[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:10:44 dist-test-slave-m0z6 krb5kdc[13143](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:10:44 dist-test-slave-m0z6 krb5kdc[13143](info): set up 2 sockets
Sep 24 23:10:44 dist-test-slave-m0z6 krb5kdc[13143](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260924 23:10:44.991183 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.952s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:10:45 dist-test-slave-m0z6 krb5kdc[13143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:10:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:10:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:10:45.109661 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:34297
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:34297
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:45.192593 13156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:45.192836 13156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:45.192870 13156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:45.194065 13156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:10:45.194110 13156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:45.194129 13156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:45.194152 13156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:10:45.194167 13156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:10:45.195760 13156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:34297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:34297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:45.196023 13156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:45.196233 13156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:45.201159 13162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:45.201282 13156 server_base.cc:1034] running on GCE node
W20260924 23:10:45.201167 13161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:45.201370 13164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:45.201640 13156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:45.201934 13156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:45.203114 13156 hybrid_clock.cc:648] HybridClock initialized: now 1790291445203091 us; error 40 us; skew 500 ppm
Sep 24 23:10:45 dist-test-slave-m0z6 krb5kdc[13143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291445, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:45.205617 13156 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:10:45.206080 13156 webserver.cc:460] Webserver started at http://127.12.212.62:42425/ using document root <none> and password file <none>
I20260924 23:10:45.206272 13156 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:45.206312 13156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:45.206410 13156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:45.207561 13156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "379efd6406ae4e32a07520533afe7d5a"
format_stamp: "Formatted at 2026-09-24 23:10:45 on dist-test-slave-m0z6"
server_key: "1612e0a55cd046f546a07629d61114dc"
server_key_iv: "800376287881b211ae0a824036292631"
server_key_version: "encryptionkey@0"
I20260924 23:10:45.207876 13156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "379efd6406ae4e32a07520533afe7d5a"
format_stamp: "Formatted at 2026-09-24 23:10:45 on dist-test-slave-m0z6"
server_key: "1612e0a55cd046f546a07629d61114dc"
server_key_iv: "800376287881b211ae0a824036292631"
server_key_version: "encryptionkey@0"
I20260924 23:10:45.209296 13156 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 23:10:45.210096 13171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:45.210260 13156 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 23:10:45.210341 13156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "379efd6406ae4e32a07520533afe7d5a"
format_stamp: "Formatted at 2026-09-24 23:10:45 on dist-test-slave-m0z6"
server_key: "1612e0a55cd046f546a07629d61114dc"
server_key_iv: "800376287881b211ae0a824036292631"
server_key_version: "encryptionkey@0"
I20260924 23:10:45.210402 13156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:45.232347 13156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:45.234213 13156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:45.234332 13156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:45.238191 13156 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:34297
I20260924 23:10:45.238230 13223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:34297 every 8 connection(s)
I20260924 23:10:45.238571 13156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:10:45.239552 13224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:45.241950 13224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a: Bootstrap starting.
I20260924 23:10:45.242556 13224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:45.242848 13224 log.cc:824] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a: Log is configured to *not* fsync() on all Append() calls
I20260924 23:10:45.243682 13224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a: No bootstrap required, opened a new log
I20260924 23:10:45.244803 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 13156
I20260924 23:10:45.244896 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:10:45.245432 13224 raft_consensus.cc:348] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379efd6406ae4e32a07520533afe7d5a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34297 } }
I20260924 23:10:45.245587 13224 raft_consensus.cc:374] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:45.245620 13224 raft_consensus.cc:729] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 379efd6406ae4e32a07520533afe7d5a, State: Initialized, Role: FOLLOWER
I20260924 23:10:45.245605 13136 external_mini_cluster.cc:1428] Setting key 3c38ca8f76fa6cdf6c8a5c03fc3b3ef6
I20260924 23:10:45.245770 13224 consensus_queue.cc:260] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379efd6406ae4e32a07520533afe7d5a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34297 } }
I20260924 23:10:45.245849 13224 raft_consensus.cc:388] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:10:45.245882 13224 raft_consensus.cc:482] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:10:45.245931 13224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:45.246649 13224 raft_consensus.cc:504] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379efd6406ae4e32a07520533afe7d5a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34297 } }
I20260924 23:10:45.246803 13224 leader_election.cc:302] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 379efd6406ae4e32a07520533afe7d5a; no voters:
I20260924 23:10:45.246966 13224 leader_election.cc:288] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:10:45.247056 13229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:10:45.247232 13224 sys_catalog.cc:564] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:10:45.247238 13229 raft_consensus.cc:686] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a [term 1 LEADER]: Becoming Leader. State: Replica: 379efd6406ae4e32a07520533afe7d5a, State: Running, Role: LEADER
I20260924 23:10:45.247357 13229 consensus_queue.cc:237] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379efd6406ae4e32a07520533afe7d5a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34297 } }
I20260924 23:10:45.247756 13229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 379efd6406ae4e32a07520533afe7d5a. Latest consensus state: current_term: 1 leader_uuid: "379efd6406ae4e32a07520533afe7d5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379efd6406ae4e32a07520533afe7d5a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34297 } } }
I20260924 23:10:45.247830 13229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a [sys.catalog]: This master's current role is: LEADER
I20260924 23:10:45.248206 13230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "379efd6406ae4e32a07520533afe7d5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379efd6406ae4e32a07520533afe7d5a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34297 } } }
I20260924 23:10:45.248287 13230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a [sys.catalog]: This master's current role is: LEADER
W20260924 23:10:45.249469 13244 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:10:45.249531 13244 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:10:45.249580 13236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:10:45.249761 13236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:10:45.251498 13236 catalog_manager.cc:1347] Generated new cluster ID: f02d7fc854044289b82c27ad1b957426
I20260924 23:10:45.251554 13236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 23:10:45 dist-test-slave-m0z6 krb5kdc[13143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:45.261430 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:45.246624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51238 (local address 127.12.212.62:34297)
0924 23:10:45.246899 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:45.246910 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:45.247875 (+ 965us) server_negotiation.cc:407] Connection header received
0924 23:10:45.249788 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:45.249795 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:45.249993 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:45.250126 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:45.251994 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:45.252942 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:45.253972 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:45.254409 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:45.254515 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:45.257815 (+ 3300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:45.257820 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:45.257820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:45.257832 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:45.259476 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:45.259928 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:45.259930 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:45.259931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:45.260055 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:45.260316 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:45.260317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:45.260318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:45.260602 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:45.260764 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:45.261032 (+ 268us) server_negotiation.cc:299] Negotiation successful
0924 23:10:45.261170 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":130,"threads_started":1}
I20260924 23:10:45.271620 13236 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:10:45.272214 13236 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:10:45.280131 13236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 379efd6406ae4e32a07520533afe7d5a: Generated new TSK 0
I20260924 23:10:45.280349 13236 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
I20260924 23:10:45.312529 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34297
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:45.396530 13250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:45.396760 13250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:45.396795 13250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:45.398087 13250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:45.398132 13250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:45.398172 13250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:10:45.399802 13250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34297
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:45.400092 13250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:45.400324 13250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:45.405820 13255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:45.405812 13256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:45.405809 13258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:45.406026 13250 server_base.cc:1034] running on GCE node
I20260924 23:10:45.406276 13250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:45.406555 13250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:45.407727 13250 hybrid_clock.cc:648] HybridClock initialized: now 1790291445407705 us; error 35 us; skew 500 ppm
Sep 24 23:10:45 dist-test-slave-m0z6 krb5kdc[13143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291445, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:45.410472 13250 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:10:45.411000 13250 webserver.cc:460] Webserver started at http://127.12.212.1:44063/ using document root <none> and password file <none>
I20260924 23:10:45.411226 13250 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:45.411278 13250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:45.411418 13250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:45.412489 13250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "714abf31676e4aca9719aeb8b2b906d1"
format_stamp: "Formatted at 2026-09-24 23:10:45 on dist-test-slave-m0z6"
server_key: "28c99197bc906815bb42c38898183b12"
server_key_iv: "6b993cd8cc477280f5dc62e6b5b137ad"
server_key_version: "encryptionkey@0"
I20260924 23:10:45.412819 13250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "714abf31676e4aca9719aeb8b2b906d1"
format_stamp: "Formatted at 2026-09-24 23:10:45 on dist-test-slave-m0z6"
server_key: "28c99197bc906815bb42c38898183b12"
server_key_iv: "6b993cd8cc477280f5dc62e6b5b137ad"
server_key_version: "encryptionkey@0"
I20260924 23:10:45.414280 13250 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:10:45.415099 13265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:45.415243 13250 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:45.415328 13250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "714abf31676e4aca9719aeb8b2b906d1"
format_stamp: "Formatted at 2026-09-24 23:10:45 on dist-test-slave-m0z6"
server_key: "28c99197bc906815bb42c38898183b12"
server_key_iv: "6b993cd8cc477280f5dc62e6b5b137ad"
server_key_version: "encryptionkey@0"
I20260924 23:10:45.415448 13250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:45.428067 13250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:45.430280 13250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:45.430436 13250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:45.430657 13250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:10:45.431028 13250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:10:45.431064 13250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:45.431093 13250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:10:45.431119 13250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:45.436748 13250 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:42969
I20260924 23:10:45.436789 13378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:42969 every 8 connection(s)
I20260924 23:10:45.437108 13250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:10:45.438009 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 13250
I20260924 23:10:45.438102 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:10:45.438242 13136 external_mini_cluster.cc:1428] Setting key 02e3bbbd96ba423f9168e9a2b2321138
Sep 24 23:10:45 dist-test-slave-m0z6 krb5kdc[13143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291445, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.212.2@KRBTEST.COM; defaulting to no policy
I20260924 23:10:45.448626 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:45.437804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:34945 (local address 127.12.212.62:34297)
0924 23:10:45.437850 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:45.437852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:45.438086 (+ 234us) server_negotiation.cc:407] Connection header received
0924 23:10:45.438725 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:45.438726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:45.438738 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:45.438785 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:45.439764 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:45.440394 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:45.441703 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:45.442026 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:45.442112 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:45.444934 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:45.444937 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:45.444937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:45.444947 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:45.446831 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:45.447347 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:45.447348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:45.447349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:45.447501 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:45.447828 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:45.447829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:45.447829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:45.447982 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:45.448054 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:45.448410 (+ 356us) server_negotiation.cc:299] Negotiation successful
0924 23:10:45.448526 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 23:10:45.448899 13381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:45.437791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:34297 (local address 127.12.212.1:34945)
0924 23:10:45.437962 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:45.438001 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:45.438592 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:45.438865 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:45.438870 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:45.439454 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:45.439659 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:45.439664 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:45.440500 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:45.440501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:45.441587 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:45.441591 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:45.442146 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:45.442147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:45.442301 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:45.442966 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:45.442977 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:45.444756 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:45.446950 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:45.446961 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:45.446964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:45.447241 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:45.447579 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:45.447580 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:45.447581 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:45.447722 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:45.448063 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:45.448074 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:45.448236 (+ 162us) client_negotiation.cc:769] Sending connection context
0924 23:10:45.448369 (+ 133us) client_negotiation.cc:240] Negotiation successful
0924 23:10:45.448640 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":104,"threads_started":1}
I20260924 23:10:45.449501 13379 heartbeater.cc:347] Connected to a master server at 127.12.212.62:34297
I20260924 23:10:45.449728 13379 heartbeater.cc:464] Registering TS with master...
I20260924 23:10:45.449911 13379 heartbeater.cc:511] Master 127.12.212.62:34297 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.2@KRBTEST.COM" created.
I20260924 23:10:45.450850 13188 ts_manager.cc:194] Registered new tserver with Master: 714abf31676e4aca9719aeb8b2b906d1 (127.12.212.1:42969)
I20260924 23:10:45.451512 13188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:34945
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
I20260924 23:10:45.471452 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34297
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:45.557288 13384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:45.557557 13384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:45.557598 13384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:45.559080 13384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:45.559144 13384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:45.559186 13384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:10:45.560920 13384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34297
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:45.561226 13384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:45.561467 13384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:45.566751 13390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:45.566751 13392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:45.566893 13389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:45.567131 13384 server_base.cc:1034] running on GCE node
I20260924 23:10:45.567314 13384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:45.567561 13384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:45.568728 13384 hybrid_clock.cc:648] HybridClock initialized: now 1790291445568693 us; error 52 us; skew 500 ppm
Sep 24 23:10:45 dist-test-slave-m0z6 krb5kdc[13143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291445, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:45.571529 13384 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:10:45.572012 13384 webserver.cc:460] Webserver started at http://127.12.212.2:35305/ using document root <none> and password file <none>
I20260924 23:10:45.572232 13384 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:45.572280 13384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:45.572409 13384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:45.573499 13384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "8aacd5e793cf4191aa93ffeade07d91d"
format_stamp: "Formatted at 2026-09-24 23:10:45 on dist-test-slave-m0z6"
server_key: "7839a4df2797b3d123e5202e6a647375"
server_key_iv: "7c1d735c53a9d00afb17ad91116e2e32"
server_key_version: "encryptionkey@0"
I20260924 23:10:45.573845 13384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "8aacd5e793cf4191aa93ffeade07d91d"
format_stamp: "Formatted at 2026-09-24 23:10:45 on dist-test-slave-m0z6"
server_key: "7839a4df2797b3d123e5202e6a647375"
server_key_iv: "7c1d735c53a9d00afb17ad91116e2e32"
server_key_version: "encryptionkey@0"
I20260924 23:10:45.575332 13384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:10:45.576263 13399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:45.576475 13384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:10:45.576567 13384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "8aacd5e793cf4191aa93ffeade07d91d"
format_stamp: "Formatted at 2026-09-24 23:10:45 on dist-test-slave-m0z6"
server_key: "7839a4df2797b3d123e5202e6a647375"
server_key_iv: "7c1d735c53a9d00afb17ad91116e2e32"
server_key_version: "encryptionkey@0"
I20260924 23:10:45.576632 13384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:45.595361 13384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:45.597656 13384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:45.597779 13384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:45.598012 13384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:10:45.598325 13384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:10:45.598356 13384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:45.598378 13384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:10:45.598389 13384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:45.604013 13384 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:41107
I20260924 23:10:45.604053 13512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:41107 every 8 connection(s)
I20260924 23:10:45.604408 13384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:10:45.606405 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 13384
I20260924 23:10:45.606523 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:10:45.606725 13136 external_mini_cluster.cc:1428] Setting key 52138ef50dbd99fb09cf0a04404e595f
Sep 24 23:10:45 dist-test-slave-m0z6 krb5kdc[13143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291445, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.212.3@KRBTEST.COM; defaulting to no policy
I20260924 23:10:45.615660 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:45.605114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:50263 (local address 127.12.212.62:34297)
0924 23:10:45.605214 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:45.605216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:45.605465 (+ 249us) server_negotiation.cc:407] Connection header received
0924 23:10:45.606054 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:45.606055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:45.606068 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:45.606136 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:45.607417 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:45.608098 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:45.609156 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:45.609433 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:45.609510 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:45.612008 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:45.612011 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:45.612011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:45.612020 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:45.613817 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:45.614478 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:45.614479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:45.614479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:45.614554 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:45.614947 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:45.614948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:45.614949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:45.615105 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:45.615177 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:45.615510 (+ 333us) server_negotiation.cc:299] Negotiation successful
0924 23:10:45.615588 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260924 23:10:45.615836 13515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:45.605075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:34297 (local address 127.12.212.2:50263)
0924 23:10:45.605278 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:45.605338 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:45.605936 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:45.606218 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:45.606222 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:45.607037 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:45.607327 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:45.607335 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:45.608191 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:45.608192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:45.609069 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:45.609072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:45.609530 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:45.609531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:45.609644 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:45.610209 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:45.610218 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:45.611866 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:45.613959 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:45.613968 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:45.613970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:45.614278 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:45.614671 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:45.614672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:45.614672 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:45.614868 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:45.615189 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:45.615196 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:45.615352 (+ 156us) client_negotiation.cc:769] Sending connection context
0924 23:10:45.615457 (+ 105us) client_negotiation.cc:240] Negotiation successful
0924 23:10:45.615607 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":83,"threads_started":1}
I20260924 23:10:45.616673 13513 heartbeater.cc:347] Connected to a master server at 127.12.212.62:34297
I20260924 23:10:45.616838 13513 heartbeater.cc:464] Registering TS with master...
I20260924 23:10:45.617085 13513 heartbeater.cc:511] Master 127.12.212.62:34297 requested a full tablet report, sending...
I20260924 23:10:45.617472 13188 ts_manager.cc:194] Registered new tserver with Master: 8aacd5e793cf4191aa93ffeade07d91d (127.12.212.2:41107)
I20260924 23:10:45.617898 13188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:50263
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
I20260924 23:10:45.634495 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34297
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:45.716094 13518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:45.716310 13518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:45.716334 13518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:45.717530 13518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:45.717583 13518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:45.717610 13518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:10:45.719136 13518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34297
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:45.719355 13518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:45.719543 13518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:45.724475 13524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:45.724479 13523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:45.724534 13526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:45.724647 13518 server_base.cc:1034] running on GCE node
I20260924 23:10:45.724900 13518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:45.725142 13518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:45.726332 13518 hybrid_clock.cc:648] HybridClock initialized: now 1790291445726312 us; error 32 us; skew 500 ppm
Sep 24 23:10:45 dist-test-slave-m0z6 krb5kdc[13143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291445, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:45.728945 13518 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:10:45.729419 13518 webserver.cc:460] Webserver started at http://127.12.212.3:39285/ using document root <none> and password file <none>
I20260924 23:10:45.729638 13518 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:45.729683 13518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:45.729815 13518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:45.730942 13518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "bf1519f239674da28bae04a00c21fef5"
format_stamp: "Formatted at 2026-09-24 23:10:45 on dist-test-slave-m0z6"
server_key: "cbbc326e0ae39474b8ab70d5f28c9a1c"
server_key_iv: "c0d40443c918517676c7a876891a2d5c"
server_key_version: "encryptionkey@0"
I20260924 23:10:45.731284 13518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "bf1519f239674da28bae04a00c21fef5"
format_stamp: "Formatted at 2026-09-24 23:10:45 on dist-test-slave-m0z6"
server_key: "cbbc326e0ae39474b8ab70d5f28c9a1c"
server_key_iv: "c0d40443c918517676c7a876891a2d5c"
server_key_version: "encryptionkey@0"
I20260924 23:10:45.732736 13518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:10:45.733655 13533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:45.733889 13518 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:10:45.733971 13518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "bf1519f239674da28bae04a00c21fef5"
format_stamp: "Formatted at 2026-09-24 23:10:45 on dist-test-slave-m0z6"
server_key: "cbbc326e0ae39474b8ab70d5f28c9a1c"
server_key_iv: "c0d40443c918517676c7a876891a2d5c"
server_key_version: "encryptionkey@0"
I20260924 23:10:45.734048 13518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:45.748112 13518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:45.750686 13518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:45.750886 13518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:45.751096 13518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:10:45.751391 13518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:10:45.751428 13518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:45.751451 13518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:10:45.751461 13518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:45.757350 13518 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:37763
I20260924 23:10:45.757413 13646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:37763 every 8 connection(s)
I20260924 23:10:45.757746 13518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:10:45.759378 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 13518
I20260924 23:10:45.759471 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:10:45.759634 13136 external_mini_cluster.cc:1428] Setting key e196184420c9be5e92815affd8a6b036
Sep 24 23:10:45 dist-test-slave-m0z6 krb5kdc[13143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291445, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:45.767707 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:45.758470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:40191 (local address 127.12.212.62:34297)
0924 23:10:45.758533 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:45.758535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:45.758735 (+ 200us) server_negotiation.cc:407] Connection header received
0924 23:10:45.759272 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:45.759273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:45.759286 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:45.759339 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:45.760477 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:45.761056 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:45.762342 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:45.762547 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:45.762603 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:45.764866 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:45.764868 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:45.764868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:45.764876 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:45.766239 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:45.766656 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:45.766657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:45.766658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:45.766743 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:45.767001 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:45.767001 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:45.767002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:45.767116 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:45.767172 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:45.767522 (+ 350us) server_negotiation.cc:299] Negotiation successful
0924 23:10:45.767612 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 23:10:45.767866 13649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:45.758446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:34297 (local address 127.12.212.3:40191)
0924 23:10:45.758594 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:45.758633 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:45.759170 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:45.759417 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:45.759422 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:45.760102 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:45.760395 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:45.760401 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:45.761142 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:45.761143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:45.762253 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:45.762256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:45.762626 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:45.762628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:45.762803 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:45.763327 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:45.763337 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:45.764762 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:45.766341 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:45.766349 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:45.766352 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:45.766584 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:45.766798 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:45.766799 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:45.766799 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:45.766931 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:45.767172 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:45.767179 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:45.767338 (+ 159us) client_negotiation.cc:769] Sending connection context
0924 23:10:45.767430 (+ 92us) client_negotiation.cc:240] Negotiation successful
0924 23:10:45.767616 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260924 23:10:45.768464 13647 heartbeater.cc:347] Connected to a master server at 127.12.212.62:34297
I20260924 23:10:45.768581 13647 heartbeater.cc:464] Registering TS with master...
I20260924 23:10:45.768771 13647 heartbeater.cc:511] Master 127.12.212.62:34297 requested a full tablet report, sending...
I20260924 23:10:45.769150 13188 ts_manager.cc:194] Registered new tserver with Master: bf1519f239674da28bae04a00c21fef5 (127.12.212.3:37763)
I20260924 23:10:45.769531 13188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:40191
I20260924 23:10:45.771670 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 23:10:45 dist-test-slave-m0z6 krb5kdc[13143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291445, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 23:10:45 dist-test-slave-m0z6 krb5kdc[13143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291445, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.212.1@KRBTEST.COM
I20260924 23:10:45.796197 13654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:45.787871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49552 (local address 127.12.212.1:42969)
0924 23:10:45.788024 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:45.788029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:45.788039 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:10:45.788072 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:45.788074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:45.788087 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:45.788155 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:45.788784 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:45.789562 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:45.790362 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:45.790745 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:45.790825 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:45.793130 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:45.793140 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:45.793140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:45.793148 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:45.794850 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:45.795277 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:45.795282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:45.795282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:45.795348 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:45.795564 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:45.795566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:45.795567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:45.795743 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:45.795876 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:45.796029 (+ 153us) server_negotiation.cc:299] Negotiation successful
0924 23:10:45.796109 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
W20260924 23:10:45.796700 13293 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49552
Sep 24 23:10:45 dist-test-slave-m0z6 krb5kdc[13143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 23:10:45 dist-test-slave-m0z6 krb5kdc[13143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.1@KRBTEST.COM
I20260924 23:10:45.819288 13654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:45.811997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49562 (local address 127.12.212.1:42969)
0924 23:10:45.812055 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:45.812057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:45.812089 (+ 32us) server_negotiation.cc:407] Connection header received
0924 23:10:45.812159 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:45.812160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:45.812170 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:45.812219 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:45.812868 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:45.813375 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:45.814199 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:45.814416 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:45.814474 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:45.816743 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:45.816745 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:45.816745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:45.816754 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:45.818216 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:45.818552 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:45.818553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:45.818553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:45.818607 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:45.818821 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:45.818822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:45.818822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:45.818980 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:45.819074 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:45.819138 (+ 64us) server_negotiation.cc:299] Negotiation successful
0924 23:10:45.819214 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 23:10:45.819783 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 13250
I20260924 23:10:45.824079 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 13384
I20260924 23:10:45.827891 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 13518
I20260924 23:10:45.831875 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 13156
2026-09-24T23:10:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1820 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:10:45 dist-test-slave-m0z6 krb5kdc[13661](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:10:45 dist-test-slave-m0z6 krb5kdc[13661](info): set up 2 sockets
Sep 24 23:10:45 dist-test-slave-m0z6 krb5kdc[13661](info): commencing operation
krb5kdc: starting...
W20260924 23:10:46.802142 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.951s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:10:46 dist-test-slave-m0z6 krb5kdc[13661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:10:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:10:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:10:46.922714 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:38143
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:38143
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:47.003145 13674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:47.003386 13674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:47.003410 13674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:47.004670 13674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:10:47.004714 13674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:47.004727 13674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:47.004746 13674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:10:47.004756 13674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:10:47.006331 13674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:38143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:38143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:47.006558 13674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:47.006790 13674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:47.012434 13680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:47.012434 13679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:47.012434 13674 server_base.cc:1034] running on GCE node
W20260924 23:10:47.012447 13682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:47.012969 13674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:47.013276 13674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:47.014448 13674 hybrid_clock.cc:648] HybridClock initialized: now 1790291447014432 us; error 32 us; skew 500 ppm
Sep 24 23:10:47 dist-test-slave-m0z6 krb5kdc[13661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291447, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:47.017127 13674 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:10:47.017606 13674 webserver.cc:460] Webserver started at http://127.12.212.62:41927/ using document root <none> and password file <none>
I20260924 23:10:47.017829 13674 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:47.017879 13674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:47.018003 13674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:47.019150 13674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "87ada10eccab4f28911db500c4188c82"
format_stamp: "Formatted at 2026-09-24 23:10:47 on dist-test-slave-m0z6"
server_key: "ed4da5809b6cfd78e3102272c8a169ae"
server_key_iv: "210262f3e59f588199aaa3cfa69299e2"
server_key_version: "encryptionkey@0"
I20260924 23:10:47.019511 13674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "87ada10eccab4f28911db500c4188c82"
format_stamp: "Formatted at 2026-09-24 23:10:47 on dist-test-slave-m0z6"
server_key: "ed4da5809b6cfd78e3102272c8a169ae"
server_key_iv: "210262f3e59f588199aaa3cfa69299e2"
server_key_version: "encryptionkey@0"
I20260924 23:10:47.020973 13674 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:10:47.021780 13689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:47.021924 13674 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:10:47.022010 13674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "87ada10eccab4f28911db500c4188c82"
format_stamp: "Formatted at 2026-09-24 23:10:47 on dist-test-slave-m0z6"
server_key: "ed4da5809b6cfd78e3102272c8a169ae"
server_key_iv: "210262f3e59f588199aaa3cfa69299e2"
server_key_version: "encryptionkey@0"
I20260924 23:10:47.022073 13674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:47.036052 13674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:47.038069 13674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:47.038211 13674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:47.042176 13674 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:38143
I20260924 23:10:47.042291 13741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:38143 every 8 connection(s)
I20260924 23:10:47.042586 13674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:10:47.043537 13742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:47.045775 13742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82: Bootstrap starting.
I20260924 23:10:47.046343 13742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:47.046607 13742 log.cc:824] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82: Log is configured to *not* fsync() on all Append() calls
I20260924 23:10:47.047420 13742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82: No bootstrap required, opened a new log
I20260924 23:10:47.047746 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 13674
I20260924 23:10:47.047828 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:10:47.047972 13136 external_mini_cluster.cc:1428] Setting key c7678faab146d752c93a0858e28b4384
I20260924 23:10:47.049449 13742 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ada10eccab4f28911db500c4188c82" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38143 } }
I20260924 23:10:47.049646 13742 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:47.049685 13742 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87ada10eccab4f28911db500c4188c82, State: Initialized, Role: FOLLOWER
I20260924 23:10:47.049817 13742 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ada10eccab4f28911db500c4188c82" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38143 } }
I20260924 23:10:47.049891 13742 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:10:47.049914 13742 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:10:47.049960 13742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:47.051186 13742 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ada10eccab4f28911db500c4188c82" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38143 } }
I20260924 23:10:47.051312 13742 leader_election.cc:302] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87ada10eccab4f28911db500c4188c82; no voters:
I20260924 23:10:47.051476 13742 leader_election.cc:288] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:10:47.051527 13747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:10:47.051716 13747 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82 [term 1 LEADER]: Becoming Leader. State: Replica: 87ada10eccab4f28911db500c4188c82, State: Running, Role: LEADER
I20260924 23:10:47.051715 13742 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:10:47.051841 13747 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ada10eccab4f28911db500c4188c82" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38143 } }
I20260924 23:10:47.052173 13747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87ada10eccab4f28911db500c4188c82. Latest consensus state: current_term: 1 leader_uuid: "87ada10eccab4f28911db500c4188c82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ada10eccab4f28911db500c4188c82" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38143 } } }
I20260924 23:10:47.052242 13747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82 [sys.catalog]: This master's current role is: LEADER
W20260924 23:10:47.053483 13763 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:10:47.053526 13763 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:10:47.053795 13748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87ada10eccab4f28911db500c4188c82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ada10eccab4f28911db500c4188c82" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38143 } } }
I20260924 23:10:47.053856 13748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82 [sys.catalog]: This master's current role is: LEADER
I20260924 23:10:47.053979 13754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:10:47.054159 13754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 23:10:47 dist-test-slave-m0z6 krb5kdc[13661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:47.055910 13754 catalog_manager.cc:1347] Generated new cluster ID: 794295642ffd4181845f600bdb03d594
I20260924 23:10:47.055967 13754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:10:47.059171 13746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.048549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39504 (local address 127.12.212.62:38143)
0924 23:10:47.048794 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:47.048805 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:47.048840 (+ 35us) server_negotiation.cc:407] Connection header received
0924 23:10:47.049398 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:47.049408 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:47.049479 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:47.049634 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:47.050364 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.051796 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.052819 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.053365 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.053465 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.056099 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:47.056103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:47.056103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:47.056113 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:47.057658 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:47.057989 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:47.057992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:47.057993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:47.058104 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:47.058287 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:47.058287 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:47.058288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:47.058547 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:47.058763 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:47.058839 (+ 76us) server_negotiation.cc:299] Negotiation successful
0924 23:10:47.058985 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":134,"threads_started":1}
I20260924 23:10:47.072376 13754 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:10:47.073159 13754 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:10:47.081799 13754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87ada10eccab4f28911db500c4188c82: Generated new TSK 0
I20260924 23:10:47.082046 13754 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
I20260924 23:10:47.112988 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:38143
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:47.194653 13768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:47.194907 13768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:47.194931 13768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:47.196092 13768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:47.196130 13768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:47.196157 13768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:10:47.197680 13768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:38143
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:47.197883 13768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:47.198069 13768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:47.203202 13776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:47.203202 13774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:47.203274 13768 server_base.cc:1034] running on GCE node
W20260924 23:10:47.203202 13773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:47.203590 13768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:47.203853 13768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:47.204990 13768 hybrid_clock.cc:648] HybridClock initialized: now 1790291447204979 us; error 30 us; skew 500 ppm
Sep 24 23:10:47 dist-test-slave-m0z6 krb5kdc[13661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291447, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:47.207582 13768 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:10:47.208118 13768 webserver.cc:460] Webserver started at http://127.12.212.1:44737/ using document root <none> and password file <none>
I20260924 23:10:47.208348 13768 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:47.208395 13768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:47.208518 13768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:47.209566 13768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "f4578b1fe1b24f51bb45ec2d98ae5a0c"
format_stamp: "Formatted at 2026-09-24 23:10:47 on dist-test-slave-m0z6"
server_key: "23fd5c0eeaf6309a6898ccc0c7615a76"
server_key_iv: "21bd7298fcd1e4334aac40d55c88bea3"
server_key_version: "encryptionkey@0"
I20260924 23:10:47.209901 13768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "f4578b1fe1b24f51bb45ec2d98ae5a0c"
format_stamp: "Formatted at 2026-09-24 23:10:47 on dist-test-slave-m0z6"
server_key: "23fd5c0eeaf6309a6898ccc0c7615a76"
server_key_iv: "21bd7298fcd1e4334aac40d55c88bea3"
server_key_version: "encryptionkey@0"
I20260924 23:10:47.211328 13768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260924 23:10:47.212121 13783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:47.212302 13768 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:10:47.212385 13768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "f4578b1fe1b24f51bb45ec2d98ae5a0c"
format_stamp: "Formatted at 2026-09-24 23:10:47 on dist-test-slave-m0z6"
server_key: "23fd5c0eeaf6309a6898ccc0c7615a76"
server_key_iv: "21bd7298fcd1e4334aac40d55c88bea3"
server_key_version: "encryptionkey@0"
I20260924 23:10:47.212459 13768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:47.232542 13768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:47.234683 13768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:47.234902 13768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:47.235117 13768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:10:47.235440 13768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:10:47.235479 13768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:47.235510 13768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:10:47.235533 13768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:47.241196 13768 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:41221
I20260924 23:10:47.241240 13896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:41221 every 8 connection(s)
I20260924 23:10:47.241564 13768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:10:47.247747 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 13768
I20260924 23:10:47.247828 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:10:47.247972 13136 external_mini_cluster.cc:1428] Setting key 09d77624c0dc1ab042b2e6eaed4b705c
Sep 24 23:10:47 dist-test-slave-m0z6 krb5kdc[13661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291447, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:47.251209 13746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.242336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:52559 (local address 127.12.212.62:38143)
0924 23:10:47.242381 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:47.242383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:47.242551 (+ 168us) server_negotiation.cc:407] Connection header received
0924 23:10:47.243536 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:47.243538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:47.243549 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:47.243616 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:47.244303 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.244823 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.245898 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.246113 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.246173 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.248497 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:47.248499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:47.248499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:47.248507 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:47.249813 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:47.250211 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:47.250212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:47.250212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:47.250267 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:47.250498 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:47.250499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:47.250499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:47.250615 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:47.250675 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:47.251015 (+ 340us) server_negotiation.cc:299] Negotiation successful
0924 23:10:47.251106 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 23:10:47.251379 13899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.242251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:38143 (local address 127.12.212.1:52559)
0924 23:10:47.242422 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:47.242468 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:47.243418 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:47.243698 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:47.243702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:47.243975 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:47.244183 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:47.244188 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.244925 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.244926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:47.245791 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:47.245793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.246193 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.246194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:47.246322 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.246941 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:47.246951 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:47.248409 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:47.249892 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:47.249901 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:47.249904 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:47.250142 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:47.250320 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:47.250321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:47.250322 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:47.250444 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:47.250673 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:47.250680 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:47.250888 (+ 208us) client_negotiation.cc:769] Sending connection context
0924 23:10:47.250984 (+ 96us) client_negotiation.cc:240] Negotiation successful
0924 23:10:47.251152 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260924 23:10:47.251809 13897 heartbeater.cc:347] Connected to a master server at 127.12.212.62:38143
I20260924 23:10:47.251911 13897 heartbeater.cc:464] Registering TS with master...
I20260924 23:10:47.252089 13897 heartbeater.cc:511] Master 127.12.212.62:38143 requested a full tablet report, sending...
I20260924 23:10:47.252575 13706 ts_manager.cc:194] Registered new tserver with Master: f4578b1fe1b24f51bb45ec2d98ae5a0c (127.12.212.1:41221)
I20260924 23:10:47.253224 13706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:52559
WARNING: no policy specified for kudu/127.12.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
I20260924 23:10:47.273046 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:38143
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:47.353861 13902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:47.354072 13902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:47.354095 13902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:47.355304 13902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:47.355351 13902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:47.355379 13902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:10:47.356861 13902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:38143
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:47.357087 13902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:47.357322 13902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:47.362066 13910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:47.362188 13908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:47.362210 13907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:47.362505 13902 server_base.cc:1034] running on GCE node
I20260924 23:10:47.362754 13902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:47.362984 13902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:47.364176 13902 hybrid_clock.cc:648] HybridClock initialized: now 1790291447364163 us; error 26 us; skew 500 ppm
Sep 24 23:10:47 dist-test-slave-m0z6 krb5kdc[13661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291447, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:47.366681 13902 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:10:47.367143 13902 webserver.cc:460] Webserver started at http://127.12.212.2:42589/ using document root <none> and password file <none>
I20260924 23:10:47.367340 13902 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:47.367389 13902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:47.367515 13902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:47.368571 13902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "d7f4d5304cd24f078b904aa55586ca4f"
format_stamp: "Formatted at 2026-09-24 23:10:47 on dist-test-slave-m0z6"
server_key: "99e767dbe64aa0a8515e2bb0f31f2c0e"
server_key_iv: "01d4664f64edaf8038475c010f952944"
server_key_version: "encryptionkey@0"
I20260924 23:10:47.368904 13902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "d7f4d5304cd24f078b904aa55586ca4f"
format_stamp: "Formatted at 2026-09-24 23:10:47 on dist-test-slave-m0z6"
server_key: "99e767dbe64aa0a8515e2bb0f31f2c0e"
server_key_iv: "01d4664f64edaf8038475c010f952944"
server_key_version: "encryptionkey@0"
I20260924 23:10:47.370137 13902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260924 23:10:47.370857 13917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:47.371037 13902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:10:47.371129 13902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "d7f4d5304cd24f078b904aa55586ca4f"
format_stamp: "Formatted at 2026-09-24 23:10:47 on dist-test-slave-m0z6"
server_key: "99e767dbe64aa0a8515e2bb0f31f2c0e"
server_key_iv: "01d4664f64edaf8038475c010f952944"
server_key_version: "encryptionkey@0"
I20260924 23:10:47.371200 13902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:47.387993 13902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:47.390206 13902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:47.390345 13902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:47.390560 13902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:10:47.390908 13902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:10:47.390944 13902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:47.390974 13902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:10:47.390991 13902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:47.396435 13902 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:40711
I20260924 23:10:47.396495 14030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:40711 every 8 connection(s)
I20260924 23:10:47.396821 13902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:10:47.397927 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 13902
I20260924 23:10:47.398010 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:10:47.398169 13136 external_mini_cluster.cc:1428] Setting key b3cd4df1cc608a827b74019ad9350624
Sep 24 23:10:47 dist-test-slave-m0z6 krb5kdc[13661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291447, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.212.3@KRBTEST.COM; defaulting to no policy
I20260924 23:10:47.408509 13746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.397584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:52569 (local address 127.12.212.62:38143)
0924 23:10:47.397645 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:47.397647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:47.397789 (+ 142us) server_negotiation.cc:407] Connection header received
0924 23:10:47.398912 (+ 1123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:47.398912 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:47.398923 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:47.399006 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:47.399891 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.400562 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.401896 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.402184 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.402260 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.404877 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:47.404880 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:47.404880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:47.404892 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:47.406610 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:47.407286 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:47.407288 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:47.407288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:47.407348 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:47.407643 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:47.407645 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:47.407646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:47.407810 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:47.407889 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:47.408334 (+ 445us) server_negotiation.cc:299] Negotiation successful
0924 23:10:47.408437 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 23:10:47.408744 14033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.397507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:38143 (local address 127.12.212.2:52569)
0924 23:10:47.397659 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:47.397701 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:47.398801 (+ 1100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:47.399105 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:47.399110 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:47.399485 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:47.399775 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:47.399781 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.400686 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.400688 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:47.401801 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:47.401804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.402321 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.402322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:47.402450 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.403008 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:47.403017 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:47.404759 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:47.406819 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:47.406832 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:47.406835 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:47.407202 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:47.407417 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:47.407418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:47.407419 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:47.407546 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:47.407902 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:47.407911 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:47.408076 (+ 165us) client_negotiation.cc:769] Sending connection context
0924 23:10:47.408237 (+ 161us) client_negotiation.cc:240] Negotiation successful
0924 23:10:47.408478 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
I20260924 23:10:47.409313 14031 heartbeater.cc:347] Connected to a master server at 127.12.212.62:38143
I20260924 23:10:47.409423 14031 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.3@KRBTEST.COM" created.
I20260924 23:10:47.409631 14031 heartbeater.cc:511] Master 127.12.212.62:38143 requested a full tablet report, sending...
I20260924 23:10:47.410043 13706 ts_manager.cc:194] Registered new tserver with Master: d7f4d5304cd24f078b904aa55586ca4f (127.12.212.2:40711)
I20260924 23:10:47.410539 13706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:52569
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
I20260924 23:10:47.425145 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:38143
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:47.506170 14036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:47.506404 14036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:47.506431 14036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:47.507994 14036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:47.508088 14036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:47.508124 14036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:10:47.510118 14036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:38143
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:47.510385 14036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:47.510592 14036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:47.515882 14042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:47.515885 14044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:47.516000 14036 server_base.cc:1034] running on GCE node
W20260924 23:10:47.515882 14041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:47.516398 14036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:47.516706 14036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:47.517879 14036 hybrid_clock.cc:648] HybridClock initialized: now 1790291447517863 us; error 30 us; skew 500 ppm
Sep 24 23:10:47 dist-test-slave-m0z6 krb5kdc[13661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291447, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:47.520605 14036 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:10:47.521102 14036 webserver.cc:460] Webserver started at http://127.12.212.3:45233/ using document root <none> and password file <none>
I20260924 23:10:47.521314 14036 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:47.521364 14036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:47.521492 14036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:47.522611 14036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "af122139015742908bb55cb119c4119f"
format_stamp: "Formatted at 2026-09-24 23:10:47 on dist-test-slave-m0z6"
server_key: "12bfb9e9a979785307266ea1341b02d6"
server_key_iv: "13acfa854d92daec700b4ce8b184a481"
server_key_version: "encryptionkey@0"
I20260924 23:10:47.523015 14036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "af122139015742908bb55cb119c4119f"
format_stamp: "Formatted at 2026-09-24 23:10:47 on dist-test-slave-m0z6"
server_key: "12bfb9e9a979785307266ea1341b02d6"
server_key_iv: "13acfa854d92daec700b4ce8b184a481"
server_key_version: "encryptionkey@0"
I20260924 23:10:47.524395 14036 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:10:47.525270 14051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:47.525461 14036 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:10:47.525538 14036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "af122139015742908bb55cb119c4119f"
format_stamp: "Formatted at 2026-09-24 23:10:47 on dist-test-slave-m0z6"
server_key: "12bfb9e9a979785307266ea1341b02d6"
server_key_iv: "13acfa854d92daec700b4ce8b184a481"
server_key_version: "encryptionkey@0"
I20260924 23:10:47.525614 14036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:47.537621 14036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:47.540560 14036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:47.540702 14036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:47.540885 14036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:10:47.541162 14036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:10:47.541191 14036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:47.541220 14036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:10:47.541236 14036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:47.546656 14036 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:37255
I20260924 23:10:47.546814 14164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:37255 every 8 connection(s)
I20260924 23:10:47.547072 14036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:10:47.550077 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 14036
I20260924 23:10:47.550184 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:10:47.550346 13136 external_mini_cluster.cc:1428] Setting key 389593c3835352792d0c448b1e3128fc
Sep 24 23:10:47 dist-test-slave-m0z6 krb5kdc[13661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291447, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:47.557379 13746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.547830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:43841 (local address 127.12.212.62:38143)
0924 23:10:47.547892 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:47.547893 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:47.548069 (+ 176us) server_negotiation.cc:407] Connection header received
0924 23:10:47.548969 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:47.548970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:47.548982 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:47.549047 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:47.549740 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.550473 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.551763 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.551968 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.552038 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.554736 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:47.554737 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:47.554738 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:47.554746 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:47.556019 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:47.556445 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:47.556446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:47.556447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:47.556501 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:47.556742 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:47.556743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:47.556744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:47.556858 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:47.556913 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:47.557202 (+ 289us) server_negotiation.cc:299] Negotiation successful
0924 23:10:47.557284 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 23:10:47.557601 14167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.547803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:38143 (local address 127.12.212.3:43841)
0924 23:10:47.547948 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:47.547990 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:47.548876 (+ 886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:47.549114 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:47.549118 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:47.549385 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:47.549591 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:47.549596 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.550590 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.550591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:47.551683 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:47.551687 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.552105 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.552106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:47.552297 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.552944 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:47.552956 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:47.554582 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:47.556108 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:47.556119 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:47.556122 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:47.556372 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:47.556555 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:47.556556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:47.556556 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:47.556687 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:47.556912 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:47.556920 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:47.557075 (+ 155us) client_negotiation.cc:769] Sending connection context
0924 23:10:47.557173 (+ 98us) client_negotiation.cc:240] Negotiation successful
0924 23:10:47.557360 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260924 23:10:47.558147 14165 heartbeater.cc:347] Connected to a master server at 127.12.212.62:38143
I20260924 23:10:47.558265 14165 heartbeater.cc:464] Registering TS with master...
I20260924 23:10:47.558461 14165 heartbeater.cc:511] Master 127.12.212.62:38143 requested a full tablet report, sending...
I20260924 23:10:47.558940 13706 ts_manager.cc:194] Registered new tserver with Master: af122139015742908bb55cb119c4119f (127.12.212.3:37255)
I20260924 23:10:47.559335 13706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:43841
I20260924 23:10:47.562197 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 23:10:47.569756 13746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.563717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39514 (local address 127.12.212.62:38143)
0924 23:10:47.563774 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:47.563776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:47.563864 (+ 88us) server_negotiation.cc:407] Connection header received
0924 23:10:47.563930 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:47.563931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:47.563941 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:47.563987 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:47.564534 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.565021 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.565727 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.565910 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.565967 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.567306 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:47.567308 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:47.567309 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:47.567318 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:47.568635 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:47.568973 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:47.568974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:47.568974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:47.569027 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:47.569200 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:47.569200 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:47.569201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:47.569357 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:47.569445 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:47.569572 (+ 127us) server_negotiation.cc:299] Negotiation successful
0924 23:10:47.569660 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 23:10:47.571985 13706 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:10:47.573650 13706 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:10:47.580492 14177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.576124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34440 (local address 127.12.212.3:37255)
0924 23:10:47.576295 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:47.576300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:47.576499 (+ 199us) server_negotiation.cc:407] Connection header received
0924 23:10:47.576550 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:47.576552 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:47.576566 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:47.576640 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:10:47.576834 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.577977 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.579329 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.580096 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.580165 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.580235 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:47.580353 (+ 118us) server_negotiation.cc:299] Negotiation successful
0924 23:10:47.580396 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":65,"threads_started":1}
I20260924 23:10:47.580488 14173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.575931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:37255 (local address 127.0.0.1:34440)
0924 23:10:47.576419 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:47.576429 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:47.576494 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:47.576649 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:47.576650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:47.576652 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:10:47.576721 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:47.576723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.578136 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.578138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:47.579238 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:47.579241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.580161 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.580162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:47.580241 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.580257 (+ 16us) client_negotiation.cc:769] Sending connection context
0924 23:10:47.580337 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 23:10:47.580396 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":18,"thread_start_us":36,"threads_started":1}
I20260924 23:10:47.581212 14174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.575878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:41221 (local address 127.0.0.1:33402)
0924 23:10:47.576249 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:47.576258 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:47.576357 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:47.576622 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:47.576625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:47.576628 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:10:47.576720 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:47.576723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.578830 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.578832 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:47.580032 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:47.580035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.580930 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.580931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:47.581009 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.581014 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 23:10:47.581080 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 23:10:47.581140 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":31,"thread_start_us":71,"threads_started":1}
I20260924 23:10:47.581516 14176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.576052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33402 (local address 127.12.212.1:41221)
0924 23:10:47.576254 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:47.576261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:47.576360 (+ 99us) server_negotiation.cc:407] Connection header received
0924 23:10:47.576426 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:47.576428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:47.576443 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:47.576513 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:10:47.576833 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.577977 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.580135 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.580849 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.580924 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.580983 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:47.581400 (+ 417us) server_negotiation.cc:299] Negotiation successful
0924 23:10:47.581451 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":105,"threads_started":1}
I20260924 23:10:47.581418 14099 tablet_service.cc:1467] Processing CreateTablet for tablet dcd14c2373194759bb06613f46cea22f (DEFAULT_TABLE table=test-table [id=aa8f69ee000849b496af7a38c4f14d10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:47.581599 14175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.575877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:40711 (local address 127.0.0.1:53712)
0924 23:10:47.576044 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:47.576091 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:47.576151 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:47.576655 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:47.576656 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:47.576658 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:10:47.576721 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:47.576724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.579031 (+ 2307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.579032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:47.580081 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:47.580082 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.581256 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.581257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:47.581435 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.581440 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 23:10:47.581491 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 23:10:47.581535 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":3,"thread_start_us":56,"threads_started":1}
I20260924 23:10:47.581833 14178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.576270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53712 (local address 127.12.212.2:40711)
0924 23:10:47.576394 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:47.576397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:47.576407 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:10:47.576451 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:47.576453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:47.576469 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:47.576542 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:10:47.578112 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.578920 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.580161 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.581110 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.581173 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.581222 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:47.581698 (+ 476us) server_negotiation.cc:299] Negotiation successful
0924 23:10:47.581761 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":69,"threads_started":1}
I20260924 23:10:47.581830 14099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcd14c2373194759bb06613f46cea22f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:47.582170 13831 tablet_service.cc:1467] Processing CreateTablet for tablet dcd14c2373194759bb06613f46cea22f (DEFAULT_TABLE table=test-table [id=aa8f69ee000849b496af7a38c4f14d10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:47.582515 13831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcd14c2373194759bb06613f46cea22f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:47.582561 13965 tablet_service.cc:1467] Processing CreateTablet for tablet dcd14c2373194759bb06613f46cea22f (DEFAULT_TABLE table=test-table [id=aa8f69ee000849b496af7a38c4f14d10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:47.582937 13965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcd14c2373194759bb06613f46cea22f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:47.585306 14179 tablet_bootstrap.cc:492] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c: Bootstrap starting.
I20260924 23:10:47.585886 14179 tablet_bootstrap.cc:654] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:47.585975 14181 tablet_bootstrap.cc:492] T dcd14c2373194759bb06613f46cea22f P d7f4d5304cd24f078b904aa55586ca4f: Bootstrap starting.
I20260924 23:10:47.586344 14180 tablet_bootstrap.cc:492] T dcd14c2373194759bb06613f46cea22f P af122139015742908bb55cb119c4119f: Bootstrap starting.
I20260924 23:10:47.586364 14179 log.cc:824] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c: Log is configured to *not* fsync() on all Append() calls
I20260924 23:10:47.586619 14181 tablet_bootstrap.cc:654] T dcd14c2373194759bb06613f46cea22f P d7f4d5304cd24f078b904aa55586ca4f: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:47.587011 14180 tablet_bootstrap.cc:654] T dcd14c2373194759bb06613f46cea22f P af122139015742908bb55cb119c4119f: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:47.587083 14179 tablet_bootstrap.cc:492] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c: No bootstrap required, opened a new log
I20260924 23:10:47.587146 14179 ts_tablet_manager.cc:1397] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:10:47.587210 14181 log.cc:824] T dcd14c2373194759bb06613f46cea22f P d7f4d5304cd24f078b904aa55586ca4f: Log is configured to *not* fsync() on all Append() calls
I20260924 23:10:47.587576 14180 log.cc:824] T dcd14c2373194759bb06613f46cea22f P af122139015742908bb55cb119c4119f: Log is configured to *not* fsync() on all Append() calls
I20260924 23:10:47.588061 14181 tablet_bootstrap.cc:492] T dcd14c2373194759bb06613f46cea22f P d7f4d5304cd24f078b904aa55586ca4f: No bootstrap required, opened a new log
I20260924 23:10:47.588133 14181 ts_tablet_manager.cc:1397] T dcd14c2373194759bb06613f46cea22f P d7f4d5304cd24f078b904aa55586ca4f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:10:47.588228 14180 tablet_bootstrap.cc:492] T dcd14c2373194759bb06613f46cea22f P af122139015742908bb55cb119c4119f: No bootstrap required, opened a new log
I20260924 23:10:47.588300 14180 ts_tablet_manager.cc:1397] T dcd14c2373194759bb06613f46cea22f P af122139015742908bb55cb119c4119f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260924 23:10:47.589358 14179 raft_consensus.cc:348] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f4d5304cd24f078b904aa55586ca4f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40711 } } peers { permanent_uuid: "f4578b1fe1b24f51bb45ec2d98ae5a0c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41221 } } peers { permanent_uuid: "af122139015742908bb55cb119c4119f" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37255 } }
I20260924 23:10:47.589555 14179 raft_consensus.cc:374] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:47.589589 14179 raft_consensus.cc:729] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4578b1fe1b24f51bb45ec2d98ae5a0c, State: Initialized, Role: FOLLOWER
I20260924 23:10:47.589653 14180 raft_consensus.cc:348] T dcd14c2373194759bb06613f46cea22f P af122139015742908bb55cb119c4119f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f4d5304cd24f078b904aa55586ca4f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40711 } } peers { permanent_uuid: "f4578b1fe1b24f51bb45ec2d98ae5a0c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41221 } } peers { permanent_uuid: "af122139015742908bb55cb119c4119f" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37255 } }
I20260924 23:10:47.589736 14179 consensus_queue.cc:260] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f4d5304cd24f078b904aa55586ca4f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40711 } } peers { permanent_uuid: "f4578b1fe1b24f51bb45ec2d98ae5a0c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41221 } } peers { permanent_uuid: "af122139015742908bb55cb119c4119f" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37255 } }
I20260924 23:10:47.589815 14180 raft_consensus.cc:374] T dcd14c2373194759bb06613f46cea22f P af122139015742908bb55cb119c4119f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:47.589855 14180 raft_consensus.cc:729] T dcd14c2373194759bb06613f46cea22f P af122139015742908bb55cb119c4119f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af122139015742908bb55cb119c4119f, State: Initialized, Role: FOLLOWER
I20260924 23:10:47.589979 14180 consensus_queue.cc:260] T dcd14c2373194759bb06613f46cea22f P af122139015742908bb55cb119c4119f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f4d5304cd24f078b904aa55586ca4f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40711 } } peers { permanent_uuid: "f4578b1fe1b24f51bb45ec2d98ae5a0c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41221 } } peers { permanent_uuid: "af122139015742908bb55cb119c4119f" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37255 } }
I20260924 23:10:47.590001 14179 ts_tablet_manager.cc:1428] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:10:47.590003 14181 raft_consensus.cc:348] T dcd14c2373194759bb06613f46cea22f P d7f4d5304cd24f078b904aa55586ca4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f4d5304cd24f078b904aa55586ca4f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40711 } } peers { permanent_uuid: "f4578b1fe1b24f51bb45ec2d98ae5a0c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41221 } } peers { permanent_uuid: "af122139015742908bb55cb119c4119f" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37255 } }
I20260924 23:10:47.590159 14181 raft_consensus.cc:374] T dcd14c2373194759bb06613f46cea22f P d7f4d5304cd24f078b904aa55586ca4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:47.590174 13897 heartbeater.cc:503] Master 127.12.212.62:38143 was elected leader, sending a full tablet report...
I20260924 23:10:47.590205 14181 raft_consensus.cc:729] T dcd14c2373194759bb06613f46cea22f P d7f4d5304cd24f078b904aa55586ca4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f4d5304cd24f078b904aa55586ca4f, State: Initialized, Role: FOLLOWER
I20260924 23:10:47.590231 14180 ts_tablet_manager.cc:1428] T dcd14c2373194759bb06613f46cea22f P af122139015742908bb55cb119c4119f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 23:10:47.590351 14165 heartbeater.cc:503] Master 127.12.212.62:38143 was elected leader, sending a full tablet report...
I20260924 23:10:47.590350 14181 consensus_queue.cc:260] T dcd14c2373194759bb06613f46cea22f P d7f4d5304cd24f078b904aa55586ca4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f4d5304cd24f078b904aa55586ca4f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40711 } } peers { permanent_uuid: "f4578b1fe1b24f51bb45ec2d98ae5a0c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41221 } } peers { permanent_uuid: "af122139015742908bb55cb119c4119f" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37255 } }
I20260924 23:10:47.590642 14181 ts_tablet_manager.cc:1428] T dcd14c2373194759bb06613f46cea22f P d7f4d5304cd24f078b904aa55586ca4f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 23:10:47.590749 14031 heartbeater.cc:503] Master 127.12.212.62:38143 was elected leader, sending a full tablet report...
I20260924 23:10:47.610074 14185 raft_consensus.cc:482] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:10:47.610212 14185 raft_consensus.cc:504] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f4d5304cd24f078b904aa55586ca4f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40711 } } peers { permanent_uuid: "f4578b1fe1b24f51bb45ec2d98ae5a0c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41221 } } peers { permanent_uuid: "af122139015742908bb55cb119c4119f" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37255 } }
I20260924 23:10:47.610612 14185 leader_election.cc:288] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7f4d5304cd24f078b904aa55586ca4f (127.12.212.2:40711), af122139015742908bb55cb119c4119f (127.12.212.3:37255)
I20260924 23:10:47.614446 14177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.610916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:44673 (local address 127.12.212.3:37255)
0924 23:10:47.610973 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:47.610975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:47.611114 (+ 139us) server_negotiation.cc:407] Connection header received
0924 23:10:47.611145 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:47.611146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:47.611157 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:47.611203 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:10:47.611327 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.612030 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.613485 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.614058 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.614133 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.614143 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:47.614327 (+ 184us) server_negotiation.cc:299] Negotiation successful
0924 23:10:47.614375 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 23:10:47.614446 14178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.610963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:48857 (local address 127.12.212.2:40711)
0924 23:10:47.611002 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:47.611004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:47.611012 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:10:47.611041 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:47.611042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:47.611049 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:47.611116 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:10:47.611287 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.612060 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.613482 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.614058 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.614133 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.614143 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:47.614327 (+ 184us) server_negotiation.cc:299] Negotiation successful
0924 23:10:47.614375 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 23:10:47.614446 13899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.610862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:40711 (local address 127.12.212.1:48857)
0924 23:10:47.610927 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:47.610944 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:47.610990 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:47.611132 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:47.611133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:47.611134 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:10:47.611215 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:47.611217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.612133 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.612135 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:47.613392 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:47.613395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.614136 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.614137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:47.614229 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.614232 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:10:47.614288 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 23:10:47.614347 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":5}
I20260924 23:10:47.614454 14188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.610862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:37255 (local address 127.12.212.1:44673)
0924 23:10:47.611048 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:47.611056 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:47.611104 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:47.611216 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:47.611217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:47.611218 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:10:47.611275 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:47.611276 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.612172 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.612173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:47.613392 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:47.613395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.614136 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.614137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:47.614228 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.614232 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:10:47.614288 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 23:10:47.614347 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":27,"thread_start_us":115,"threads_started":1}
I20260924 23:10:47.614872 13985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcd14c2373194759bb06613f46cea22f" candidate_uuid: "f4578b1fe1b24f51bb45ec2d98ae5a0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f4d5304cd24f078b904aa55586ca4f" is_pre_election: true
I20260924 23:10:47.614869 14119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcd14c2373194759bb06613f46cea22f" candidate_uuid: "f4578b1fe1b24f51bb45ec2d98ae5a0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af122139015742908bb55cb119c4119f" is_pre_election: true
I20260924 23:10:47.615039 13985 raft_consensus.cc:2454] T dcd14c2373194759bb06613f46cea22f P d7f4d5304cd24f078b904aa55586ca4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4578b1fe1b24f51bb45ec2d98ae5a0c in term 0.
I20260924 23:10:47.615061 14119 raft_consensus.cc:2454] T dcd14c2373194759bb06613f46cea22f P af122139015742908bb55cb119c4119f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4578b1fe1b24f51bb45ec2d98ae5a0c in term 0.
I20260924 23:10:47.615278 13787 leader_election.cc:302] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: af122139015742908bb55cb119c4119f, d7f4d5304cd24f078b904aa55586ca4f, f4578b1fe1b24f51bb45ec2d98ae5a0c; no voters:
I20260924 23:10:47.615437 14185 raft_consensus.cc:2781] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:10:47.615492 14185 raft_consensus.cc:482] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:10:47.615509 14185 raft_consensus.cc:3037] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:47.616227 14185 raft_consensus.cc:504] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f4d5304cd24f078b904aa55586ca4f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40711 } } peers { permanent_uuid: "f4578b1fe1b24f51bb45ec2d98ae5a0c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41221 } } peers { permanent_uuid: "af122139015742908bb55cb119c4119f" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37255 } }
I20260924 23:10:47.616348 14185 leader_election.cc:288] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c [CANDIDATE]: Term 1 election: Requested vote from peers d7f4d5304cd24f078b904aa55586ca4f (127.12.212.2:40711), af122139015742908bb55cb119c4119f (127.12.212.3:37255)
I20260924 23:10:47.616520 13985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcd14c2373194759bb06613f46cea22f" candidate_uuid: "f4578b1fe1b24f51bb45ec2d98ae5a0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f4d5304cd24f078b904aa55586ca4f"
I20260924 23:10:47.616605 13985 raft_consensus.cc:3037] T dcd14c2373194759bb06613f46cea22f P d7f4d5304cd24f078b904aa55586ca4f [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:47.617342 13985 raft_consensus.cc:2454] T dcd14c2373194759bb06613f46cea22f P d7f4d5304cd24f078b904aa55586ca4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4578b1fe1b24f51bb45ec2d98ae5a0c in term 1.
I20260924 23:10:47.617452 14119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcd14c2373194759bb06613f46cea22f" candidate_uuid: "f4578b1fe1b24f51bb45ec2d98ae5a0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af122139015742908bb55cb119c4119f"
I20260924 23:10:47.617534 13785 leader_election.cc:302] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7f4d5304cd24f078b904aa55586ca4f, f4578b1fe1b24f51bb45ec2d98ae5a0c; no voters:
I20260924 23:10:47.617527 14119 raft_consensus.cc:3037] T dcd14c2373194759bb06613f46cea22f P af122139015742908bb55cb119c4119f [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:47.617632 14185 raft_consensus.cc:2781] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:10:47.617784 14185 raft_consensus.cc:686] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c [term 1 LEADER]: Becoming Leader. State: Replica: f4578b1fe1b24f51bb45ec2d98ae5a0c, State: Running, Role: LEADER
I20260924 23:10:47.617887 14185 consensus_queue.cc:237] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f4d5304cd24f078b904aa55586ca4f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40711 } } peers { permanent_uuid: "f4578b1fe1b24f51bb45ec2d98ae5a0c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41221 } } peers { permanent_uuid: "af122139015742908bb55cb119c4119f" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37255 } }
I20260924 23:10:47.618347 14119 raft_consensus.cc:2454] T dcd14c2373194759bb06613f46cea22f P af122139015742908bb55cb119c4119f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4578b1fe1b24f51bb45ec2d98ae5a0c in term 1.
I20260924 23:10:47.618685 13705 catalog_manager.cc:5495] T dcd14c2373194759bb06613f46cea22f P f4578b1fe1b24f51bb45ec2d98ae5a0c reported cstate change: term changed from 0 to 1, leader changed from <none> to f4578b1fe1b24f51bb45ec2d98ae5a0c (127.12.212.1). New cstate: current_term: 1 leader_uuid: "f4578b1fe1b24f51bb45ec2d98ae5a0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f4d5304cd24f078b904aa55586ca4f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4578b1fe1b24f51bb45ec2d98ae5a0c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af122139015742908bb55cb119c4119f" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37255 } health_report { overall_health: UNKNOWN } } }
Sep 24 23:10:47 dist-test-slave-m0z6 krb5kdc[13661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.1@KRBTEST.COM
I20260924 23:10:47.630425 14176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.622596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33408 (local address 127.12.212.1:41221)
0924 23:10:47.622636 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:47.622638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:47.622807 (+ 169us) server_negotiation.cc:407] Connection header received
0924 23:10:47.622869 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:47.622870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:47.622882 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:47.622931 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:47.623492 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.623974 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.624693 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.624933 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.624989 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.627474 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:47.627487 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:47.627487 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:47.627496 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:47.629243 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:47.629629 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:47.629633 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:47.629633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:47.629691 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:47.629858 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:47.629858 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:47.629859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:47.630041 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:47.630183 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:47.630245 (+ 62us) server_negotiation.cc:299] Negotiation successful
0924 23:10:47.630327 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Sep 24 23:10:47 dist-test-slave-m0z6 krb5kdc[13661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291447, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260924 23:10:47.647425 14032 tablet.cc:2369] T dcd14c2373194759bb06613f46cea22f P d7f4d5304cd24f078b904aa55586ca4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 24 23:10:47 dist-test-slave-m0z6 krb5kdc[13661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291447, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.212.1@KRBTEST.COM
I20260924 23:10:47.652837 14176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.645832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33418 (local address 127.12.212.1:41221)
0924 23:10:47.645910 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:47.645912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:47.645970 (+ 58us) server_negotiation.cc:407] Connection header received
0924 23:10:47.646045 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:47.646046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:47.646055 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:47.646111 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:47.646742 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.647229 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.647964 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.648165 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.648219 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.650477 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:47.650478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:47.650479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:47.650487 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:47.651723 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:47.652055 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:47.652057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:47.652057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:47.652109 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:47.652275 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:47.652275 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:47.652276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:47.652371 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:47.652459 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:47.652663 (+ 204us) server_negotiation.cc:299] Negotiation successful
0924 23:10:47.652746 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Sep 24 23:10:47 dist-test-slave-m0z6 krb5kdc[13661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 23:10:47 dist-test-slave-m0z6 krb5kdc[13661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.1@KRBTEST.COM
I20260924 23:10:47.675568 14176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:47.667860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33430 (local address 127.12.212.1:41221)
0924 23:10:47.667926 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:47.667928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:47.668070 (+ 142us) server_negotiation.cc:407] Connection header received
0924 23:10:47.668101 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:47.668101 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:47.668114 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:47.668157 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:47.668855 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.669390 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.670226 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:47.670422 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:47.670487 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:47.672860 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:47.672861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:47.672862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:47.672869 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:47.674291 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:47.674748 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:47.674749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:47.674749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:47.674820 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:47.675050 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:47.675051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:47.675051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:47.675165 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:47.675262 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:47.675396 (+ 134us) server_negotiation.cc:299] Negotiation successful
0924 23:10:47.675474 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260924 23:10:47.676128 13811 tablet_service.cc:2908] Rejecting scan request for tablet dcd14c2373194759bb06613f46cea22f: Uninitialized: clean time has not yet been initialized
I20260924 23:10:47.677269 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 13768
I20260924 23:10:47.681493 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 13902
I20260924 23:10:47.685760 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 14036
I20260924 23:10:47.689651 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 13674
2026-09-24T23:10:47Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1858 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:10:47 dist-test-slave-m0z6 krb5kdc[14203](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:10:47 dist-test-slave-m0z6 krb5kdc[14203](info): set up 2 sockets
Sep 24 23:10:47 dist-test-slave-m0z6 krb5kdc[14203](info): commencing operation
krb5kdc: starting...
W20260924 23:10:48.642151 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:10:48 dist-test-slave-m0z6 krb5kdc[14203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:10:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:10:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:10:48.762370 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:42649
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:38163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:42649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:48.845084 14216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:48.845314 14216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:48.845337 14216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:48.846552 14216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:10:48.846601 14216 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 23:10:48.846618 14216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:48.846635 14216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:48.846650 14216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:10:48.846674 14216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:10:48.848248 14216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:38163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:42649
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:42649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:48.848464 14216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:48.848651 14216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:48.853397 14222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:48.853398 14221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:48.853446 14224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:48.853415 14216 server_base.cc:1034] running on GCE node
I20260924 23:10:48.853811 14216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:48.854113 14216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:48.855271 14216 hybrid_clock.cc:648] HybridClock initialized: now 1790291448855251 us; error 36 us; skew 500 ppm
Sep 24 23:10:48 dist-test-slave-m0z6 krb5kdc[14203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291448, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:48.857816 14216 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:10:48.858264 14216 webserver.cc:460] Webserver started at http://127.12.212.62:39525/ using document root <none> and password file <none>
I20260924 23:10:48.858476 14216 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:48.858523 14216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:48.858654 14216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:48.859752 14216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "52a022b9db364b9384ddceb76007e38c"
format_stamp: "Formatted at 2026-09-24 23:10:48 on dist-test-slave-m0z6"
server_key: "0cf3ddfdc6dfa1c07c630d840482708f"
server_key_iv: "cca724d180263eff184d42e805132469"
server_key_version: "encryptionkey@0"
I20260924 23:10:48.860090 14216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "52a022b9db364b9384ddceb76007e38c"
format_stamp: "Formatted at 2026-09-24 23:10:48 on dist-test-slave-m0z6"
server_key: "0cf3ddfdc6dfa1c07c630d840482708f"
server_key_iv: "cca724d180263eff184d42e805132469"
server_key_version: "encryptionkey@0"
I20260924 23:10:48.861406 14216 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:10:48.862124 14231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:48.862295 14216 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:10:48.862378 14216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "52a022b9db364b9384ddceb76007e38c"
format_stamp: "Formatted at 2026-09-24 23:10:48 on dist-test-slave-m0z6"
server_key: "0cf3ddfdc6dfa1c07c630d840482708f"
server_key_iv: "cca724d180263eff184d42e805132469"
server_key_version: "encryptionkey@0"
I20260924 23:10:48.862442 14216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:48.873579 14216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:48.875583 14216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:48.875742 14216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:48.880134 14216 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:42649
I20260924 23:10:48.880190 14293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:42649 every 8 connection(s)
I20260924 23:10:48.880503 14216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:10:48.881127 14294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:48.883453 14294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c: Bootstrap starting.
I20260924 23:10:48.884047 14294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:48.884310 14294 log.cc:824] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c: Log is configured to *not* fsync() on all Append() calls
I20260924 23:10:48.885141 14294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c: No bootstrap required, opened a new log
I20260924 23:10:48.887305 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 14216
I20260924 23:10:48.887391 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:10:48.887529 13136 external_mini_cluster.cc:1428] Setting key 26d9f7d7ecf58bea564927ae2ea85aa5
I20260924 23:10:48.887388 14294 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52a022b9db364b9384ddceb76007e38c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42649 } }
I20260924 23:10:48.887570 14294 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:48.887611 14294 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52a022b9db364b9384ddceb76007e38c, State: Initialized, Role: FOLLOWER
I20260924 23:10:48.887761 14294 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52a022b9db364b9384ddceb76007e38c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42649 } }
I20260924 23:10:48.887843 14294 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:10:48.887880 14294 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:10:48.887921 14294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:48.888623 14294 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52a022b9db364b9384ddceb76007e38c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42649 } }
I20260924 23:10:48.888761 14294 leader_election.cc:302] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52a022b9db364b9384ddceb76007e38c; no voters:
I20260924 23:10:48.888947 14294 leader_election.cc:288] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:10:48.888993 14299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:10:48.889191 14299 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c [term 1 LEADER]: Becoming Leader. State: Replica: 52a022b9db364b9384ddceb76007e38c, State: Running, Role: LEADER
I20260924 23:10:48.889232 14294 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:10:48.889281 14299 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52a022b9db364b9384ddceb76007e38c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42649 } }
I20260924 23:10:48.889628 14299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52a022b9db364b9384ddceb76007e38c. Latest consensus state: current_term: 1 leader_uuid: "52a022b9db364b9384ddceb76007e38c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52a022b9db364b9384ddceb76007e38c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42649 } } }
I20260924 23:10:48.889703 14299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c [sys.catalog]: This master's current role is: LEADER
I20260924 23:10:48.889681 14300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52a022b9db364b9384ddceb76007e38c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52a022b9db364b9384ddceb76007e38c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42649 } } }
I20260924 23:10:48.889927 14300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c [sys.catalog]: This master's current role is: LEADER
W20260924 23:10:48.891089 14315 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:10:48.891155 14315 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:10:48.891198 14308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:10:48.891348 14308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:10:48.892917 14308 catalog_manager.cc:1347] Generated new cluster ID: 8bca7ada19cc433195d1a7af37c46fc8
I20260924 23:10:48.892976 14308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 23:10:48 dist-test-slave-m0z6 krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:48.899477 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:48.888002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50676 (local address 127.12.212.62:42649)
0924 23:10:48.888181 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:48.888192 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:48.888223 (+ 31us) server_negotiation.cc:407] Connection header received
0924 23:10:48.888329 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:48.888334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:48.888377 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:48.888496 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:48.889240 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:48.890726 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:48.891780 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:48.892534 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:48.892662 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:48.895573 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:48.895579 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:48.895579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:48.895595 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:48.897535 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:48.897985 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:48.897988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:48.897988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:48.898125 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:48.898348 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:48.898348 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:48.898348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:48.898658 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:48.898846 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:48.898980 (+ 134us) server_negotiation.cc:299] Negotiation successful
0924 23:10:48.899239 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260924 23:10:48.900209 14308 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:10:48.900771 14308 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:10:48.903712 14308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c: Generated new TSK 0
I20260924 23:10:48.903923 14308 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
I20260924 23:10:48.934126 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42649
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:38163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:49.018054 14320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:49.018302 14320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:49.018328 14320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:49.019660 14320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:49.019721 14320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:49.019743 14320 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 23:10:49.019761 14320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:10:49.021602 14320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:38163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.212.62:42649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:49.021878 14320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:49.022132 14320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:49.027740 14328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:49.028156 14325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:49.028169 14326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:49.028362 14320 server_base.cc:1034] running on GCE node
I20260924 23:10:49.028606 14320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:49.028873 14320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:49.030074 14320 hybrid_clock.cc:648] HybridClock initialized: now 1790291449030058 us; error 31 us; skew 500 ppm
Sep 24 23:10:49 dist-test-slave-m0z6 krb5kdc[14203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291449, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:49.032958 14320 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:10:49.033524 14320 webserver.cc:460] Webserver started at http://127.12.212.1:38197/ using document root <none> and password file <none>
I20260924 23:10:49.033751 14320 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:49.033806 14320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:49.033942 14320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:49.035280 14320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "dc45e7177ecc4d2797873dd4ed416970"
format_stamp: "Formatted at 2026-09-24 23:10:49 on dist-test-slave-m0z6"
server_key: "2447ab2e362f0ed940cd6f7d38fadf49"
server_key_iv: "036a3aa3146c581ed8afa95173621865"
server_key_version: "encryptionkey@0"
I20260924 23:10:49.035650 14320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "dc45e7177ecc4d2797873dd4ed416970"
format_stamp: "Formatted at 2026-09-24 23:10:49 on dist-test-slave-m0z6"
server_key: "2447ab2e362f0ed940cd6f7d38fadf49"
server_key_iv: "036a3aa3146c581ed8afa95173621865"
server_key_version: "encryptionkey@0"
I20260924 23:10:49.037171 14320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:10:49.038192 14335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:49.038348 14320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:10:49.038437 14320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "dc45e7177ecc4d2797873dd4ed416970"
format_stamp: "Formatted at 2026-09-24 23:10:49 on dist-test-slave-m0z6"
server_key: "2447ab2e362f0ed940cd6f7d38fadf49"
server_key_iv: "036a3aa3146c581ed8afa95173621865"
server_key_version: "encryptionkey@0"
I20260924 23:10:49.038544 14320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:49.055601 14320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:49.058491 14320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:49.058670 14320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:49.059314 14320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:10:49.059371 14320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:49.059403 14320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:10:49.059419 14320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:49.066324 14320 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:35911
I20260924 23:10:49.066377 14450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:35911 every 8 connection(s)
Sep 24 23:10:49 dist-test-slave-m0z6 krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291449, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:49.066747 14320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:10:49.069376 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 14320
I20260924 23:10:49.069499 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:10:49.069669 13136 external_mini_cluster.cc:1428] Setting key 0e6d81041c0524f36ae7455712d0f563
I20260924 23:10:49.070473 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.059777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:48639 (local address 127.12.212.62:42649)
0924 23:10:49.059840 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:49.059843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:49.060037 (+ 194us) server_negotiation.cc:407] Connection header received
0924 23:10:49.060557 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:49.060559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:49.060575 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:49.060626 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:49.061331 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.062132 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.063769 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.064104 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.064177 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.067264 (+ 3087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:49.067266 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:49.067266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:49.067273 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:49.068724 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:49.069310 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:49.069312 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:49.069313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:49.069392 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:49.069733 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:49.069734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:49.069735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:49.069890 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:49.069947 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:49.070258 (+ 311us) server_negotiation.cc:299] Negotiation successful
0924 23:10:49.070371 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 23:10:49.070750 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.059752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42649 (local address 127.12.212.1:48639)
0924 23:10:49.059905 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:49.059949 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:49.060444 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:49.060711 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:49.060715 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:49.061015 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:49.061222 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.061227 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.062231 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.062232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.063669 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.063674 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.064199 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.064201 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.064385 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.065023 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:49.065035 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:49.067162 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:49.068901 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:49.068910 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:49.068913 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:49.069197 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:49.069475 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:49.069476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:49.069476 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:49.069657 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:49.069942 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:49.069950 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:49.070127 (+ 177us) client_negotiation.cc:769] Sending connection context
0924 23:10:49.070227 (+ 100us) client_negotiation.cc:240] Negotiation successful
0924 23:10:49.070421 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":83,"threads_started":1}
I20260924 23:10:49.071239 14451 heartbeater.cc:347] Connected to a master server at 127.12.212.62:42649
I20260924 23:10:49.071331 14451 heartbeater.cc:464] Registering TS with master...
I20260924 23:10:49.071545 14451 heartbeater.cc:511] Master 127.12.212.62:42649 requested a full tablet report, sending...
I20260924 23:10:49.072085 14247 ts_manager.cc:194] Registered new tserver with Master: dc45e7177ecc4d2797873dd4ed416970 (127.12.212.1:35911)
I20260924 23:10:49.072702 14247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:48639
WARNING: no policy specified for kudu/127.12.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.2@KRBTEST.COM" created.
I20260924 23:10:49.084252 14458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.072974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42649 (local address 127.12.212.1:59361)
0924 23:10:49.073134 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:49.073142 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:49.073193 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:49.073310 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:49.073311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:49.073514 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:49.073569 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.073570 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.074227 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.074228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.074821 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.074823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.075171 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.075172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.075252 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.081172 (+ 5920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:49.081179 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:49.081518 (+ 339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:49.083154 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:49.083155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:49.083156 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:49.083347 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:49.083544 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:49.083545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:49.083545 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:49.083605 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:49.083878 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:49.083879 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:49.083932 (+ 53us) client_negotiation.cc:769] Sending connection context
0924 23:10:49.083998 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 23:10:49.084133 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260924 23:10:49.084242 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.073000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:59361 (local address 127.12.212.62:42649)
0924 23:10:49.073041 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:49.073043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:49.073196 (+ 153us) server_negotiation.cc:407] Connection header received
0924 23:10:49.073250 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:49.073251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:49.073262 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:49.073308 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:49.073640 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.074136 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.074893 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.075104 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.075161 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.081649 (+ 6488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:49.081651 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:49.081651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:49.081660 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:49.083054 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:49.083420 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:49.083422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:49.083422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:49.083485 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:49.083658 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:49.083659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:49.083659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:49.083805 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:49.083874 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:49.084035 (+ 161us) server_negotiation.cc:299] Negotiation successful
0924 23:10:49.084131 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
I20260924 23:10:49.098238 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42649
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:38163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:49.181032 14460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:49.181284 14460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:49.181311 14460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:49.182634 14460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:49.182715 14460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:49.182740 14460 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 23:10:49.182760 14460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:10:49.184338 14460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:38163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.212.62:42649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:49.184587 14460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:49.184839 14460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:49.189911 14465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:49.189924 14466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:49.190004 14468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:49.190176 14460 server_base.cc:1034] running on GCE node
I20260924 23:10:49.190402 14460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:49.190665 14460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:49.191866 14460 hybrid_clock.cc:648] HybridClock initialized: now 1790291449191849 us; error 29 us; skew 500 ppm
Sep 24 23:10:49 dist-test-slave-m0z6 krb5kdc[14203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291449, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:49.194540 14460 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:10:49.195142 14460 webserver.cc:460] Webserver started at http://127.12.212.2:41197/ using document root <none> and password file <none>
I20260924 23:10:49.195364 14460 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:49.195411 14460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:49.195549 14460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:49.196681 14460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "550ba8a5f1e24c299fd0f708c639abbc"
format_stamp: "Formatted at 2026-09-24 23:10:49 on dist-test-slave-m0z6"
server_key: "23f8104d46db241267bea24b273ace3a"
server_key_iv: "2566aa18d7e5d2bf8f49762b5798623a"
server_key_version: "encryptionkey@0"
I20260924 23:10:49.197041 14460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "550ba8a5f1e24c299fd0f708c639abbc"
format_stamp: "Formatted at 2026-09-24 23:10:49 on dist-test-slave-m0z6"
server_key: "23f8104d46db241267bea24b273ace3a"
server_key_iv: "2566aa18d7e5d2bf8f49762b5798623a"
server_key_version: "encryptionkey@0"
I20260924 23:10:49.198473 14460 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:10:49.199586 14475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:49.199805 14460 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:10:49.199965 14460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "550ba8a5f1e24c299fd0f708c639abbc"
format_stamp: "Formatted at 2026-09-24 23:10:49 on dist-test-slave-m0z6"
server_key: "23f8104d46db241267bea24b273ace3a"
server_key_iv: "2566aa18d7e5d2bf8f49762b5798623a"
server_key_version: "encryptionkey@0"
I20260924 23:10:49.200035 14460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:49.214452 14460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:49.216847 14460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:49.217006 14460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:49.217567 14460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:10:49.217613 14460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:49.217644 14460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:10:49.217660 14460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:49.223907 14460 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:40019
I20260924 23:10:49.224110 14590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:40019 every 8 connection(s)
I20260924 23:10:49.224321 14460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
Sep 24 23:10:49 dist-test-slave-m0z6 krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291449, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:49.228494 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.218037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:38757 (local address 127.12.212.62:42649)
0924 23:10:49.218101 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:49.218104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:49.218318 (+ 214us) server_negotiation.cc:407] Connection header received
0924 23:10:49.218885 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:49.218886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:49.218900 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:49.218979 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:49.219895 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.220665 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.222314 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.222633 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.222739 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.225490 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:49.225493 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:49.225493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:49.225502 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:49.227008 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:49.227415 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:49.227417 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:49.227417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:49.227476 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:49.227708 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:49.227709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:49.227710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:49.227833 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:49.227890 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:49.228308 (+ 418us) server_negotiation.cc:299] Negotiation successful
0924 23:10:49.228402 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 23:10:49.228778 14484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.218021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42649 (local address 127.12.212.2:38757)
0924 23:10:49.218189 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:49.218231 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:49.218764 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:49.219071 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:49.219076 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:49.219473 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:49.219775 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.219782 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.220770 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.220771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.222198 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.222202 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.222757 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.222758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.222927 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.223570 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:49.223579 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:49.225383 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:49.227096 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:49.227104 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:49.227107 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:49.227345 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:49.227533 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:49.227534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:49.227534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:49.227651 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:49.227889 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:49.227896 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:49.228084 (+ 188us) client_negotiation.cc:769] Sending connection context
0924 23:10:49.228257 (+ 173us) client_negotiation.cc:240] Negotiation successful
0924 23:10:49.228445 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
I20260924 23:10:49.229265 14591 heartbeater.cc:347] Connected to a master server at 127.12.212.62:42649
I20260924 23:10:49.229353 14591 heartbeater.cc:464] Registering TS with master...
I20260924 23:10:49.229583 14591 heartbeater.cc:511] Master 127.12.212.62:42649 requested a full tablet report, sending...
I20260924 23:10:49.229982 14248 ts_manager.cc:194] Registered new tserver with Master: 550ba8a5f1e24c299fd0f708c639abbc (127.12.212.2:40019)
I20260924 23:10:49.230397 14248 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:38757
I20260924 23:10:49.233355 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 14460
I20260924 23:10:49.233458 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:10:49.233641 13136 external_mini_cluster.cc:1428] Setting key 09d23a676cf10e384d9488610d10e410
I20260924 23:10:49.236176 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.230380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:37365 (local address 127.12.212.62:42649)
0924 23:10:49.230419 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:49.230420 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:49.230498 (+ 78us) server_negotiation.cc:407] Connection header received
0924 23:10:49.230574 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:49.230575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:49.230584 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:49.230625 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:49.231031 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.231498 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.232199 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.232384 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.232436 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.233308 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:49.233309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:49.233309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:49.233317 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:49.234976 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:49.235292 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:49.235293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:49.235293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:49.235353 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:49.235622 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:49.235623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:49.235624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:49.235783 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:49.235847 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:49.235999 (+ 152us) server_negotiation.cc:299] Negotiation successful
0924 23:10:49.236080 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 23:10:49.236191 14597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.230308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42649 (local address 127.12.212.2:37365)
0924 23:10:49.230430 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:49.230442 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:49.230524 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:49.230638 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:49.230639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:49.230876 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:49.230940 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.230942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.231598 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.231599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.232117 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.232118 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.232445 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.232446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.232510 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.232892 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:49.232896 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:49.233215 (+ 319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:49.235065 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:49.235066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:49.235066 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:49.235229 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:49.235468 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:49.235469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:49.235469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:49.235535 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:49.235846 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:49.235847 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:49.235899 (+ 52us) client_negotiation.cc:769] Sending connection context
0924 23:10:49.235963 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 23:10:49.236082 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
WARNING: no policy specified for kudu/127.12.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
I20260924 23:10:49.257647 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42649
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:38163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:49.340967 14600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:49.341200 14600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:49.341238 14600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:49.342574 14600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:49.342620 14600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:49.342648 14600 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 23:10:49.342672 14600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:10:49.344276 14600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:38163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.212.62:42649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:49.344545 14600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:49.344769 14600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:49.349901 14606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:49.349901 14605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:49.350047 14608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:49.350857 14600 server_base.cc:1034] running on GCE node
I20260924 23:10:49.351257 14600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:49.351536 14600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:49.352716 14600 hybrid_clock.cc:648] HybridClock initialized: now 1790291449352699 us; error 35 us; skew 500 ppm
Sep 24 23:10:49 dist-test-slave-m0z6 krb5kdc[14203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291449, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:49.355580 14600 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:10:49.356084 14600 webserver.cc:460] Webserver started at http://127.12.212.3:43903/ using document root <none> and password file <none>
I20260924 23:10:49.356303 14600 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:49.356350 14600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:49.356482 14600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:49.357574 14600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "8e60595e5ce74ff2ae43f557c216d55a"
format_stamp: "Formatted at 2026-09-24 23:10:49 on dist-test-slave-m0z6"
server_key: "7deeb94d22aec31d9b308374b4f08538"
server_key_iv: "9d035176090743daed8e8eb53f23a085"
server_key_version: "encryptionkey@0"
I20260924 23:10:49.357904 14600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "8e60595e5ce74ff2ae43f557c216d55a"
format_stamp: "Formatted at 2026-09-24 23:10:49 on dist-test-slave-m0z6"
server_key: "7deeb94d22aec31d9b308374b4f08538"
server_key_iv: "9d035176090743daed8e8eb53f23a085"
server_key_version: "encryptionkey@0"
I20260924 23:10:49.359319 14600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:10:49.360177 14615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:49.360436 14600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:10:49.360533 14600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "8e60595e5ce74ff2ae43f557c216d55a"
format_stamp: "Formatted at 2026-09-24 23:10:49 on dist-test-slave-m0z6"
server_key: "7deeb94d22aec31d9b308374b4f08538"
server_key_iv: "9d035176090743daed8e8eb53f23a085"
server_key_version: "encryptionkey@0"
I20260924 23:10:49.360602 14600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:49.372085 14600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:49.374346 14600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:49.374490 14600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:49.375131 14600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:10:49.375176 14600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:49.375207 14600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:10:49.375236 14600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:49.381942 14600 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:33329
Sep 24 23:10:49 dist-test-slave-m0z6 krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291449, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:49.382074 14730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:33329 every 8 connection(s)
I20260924 23:10:49.382344 14600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:10:49.382655 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 14600
I20260924 23:10:49.382769 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:10:49.382943 13136 external_mini_cluster.cc:1428] Setting key 57c493670884e937b11aa95e9edaaf12
I20260924 23:10:49.385526 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.375513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:59667 (local address 127.12.212.62:42649)
0924 23:10:49.375566 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:49.375568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:49.375805 (+ 237us) server_negotiation.cc:407] Connection header received
0924 23:10:49.376292 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:49.376294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:49.376306 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:49.376385 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:49.377154 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.377915 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.379545 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.379773 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.379832 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.382509 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:49.382511 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:49.382511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:49.382521 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:49.384145 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:49.384549 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:49.384551 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:49.384551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:49.384610 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:49.384860 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:49.384861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:49.384862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:49.384987 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:49.385046 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:49.385324 (+ 278us) server_negotiation.cc:299] Negotiation successful
0924 23:10:49.385421 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 23:10:49.385703 14624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.375509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42649 (local address 127.12.212.3:59667)
0924 23:10:49.375672 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:49.375715 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:49.376196 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:49.376496 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:49.376500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:49.376834 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:49.377056 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.377062 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.378031 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.378032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.379433 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.379437 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.379866 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.379867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.380036 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.380626 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:49.380638 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:49.382402 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:49.384223 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:49.384233 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:49.384235 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:49.384479 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:49.384667 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:49.384668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:49.384668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:49.384787 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:49.385041 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:49.385048 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:49.385198 (+ 150us) client_negotiation.cc:769] Sending connection context
0924 23:10:49.385293 (+ 95us) client_negotiation.cc:240] Negotiation successful
0924 23:10:49.385468 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260924 23:10:49.386202 14731 heartbeater.cc:347] Connected to a master server at 127.12.212.62:42649
I20260924 23:10:49.386308 14731 heartbeater.cc:464] Registering TS with master...
I20260924 23:10:49.386529 14731 heartbeater.cc:511] Master 127.12.212.62:42649 requested a full tablet report, sending...
I20260924 23:10:49.386951 14247 ts_manager.cc:194] Registered new tserver with Master: 8e60595e5ce74ff2ae43f557c216d55a (127.12.212.3:33329)
I20260924 23:10:49.387444 14247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:59667
I20260924 23:10:49.393075 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.387266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:53695 (local address 127.12.212.62:42649)
0924 23:10:49.387566 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:49.387568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:49.387573 (+ 5us) server_negotiation.cc:407] Connection header received
0924 23:10:49.387590 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:49.387591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:49.387599 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:49.387645 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:49.388068 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.388526 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.389257 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.389471 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.389535 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.390435 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:49.390437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:49.390437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:49.390443 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:49.391828 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:49.392221 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:49.392222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:49.392222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:49.392284 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:49.392527 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:49.392528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:49.392528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:49.392653 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:49.392722 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:49.392889 (+ 167us) server_negotiation.cc:299] Negotiation successful
0924 23:10:49.392960 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283}
I20260924 23:10:49.393076 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.387195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42649 (local address 127.12.212.3:53695)
0924 23:10:49.387310 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:49.387321 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:49.387370 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:49.387661 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:49.387662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:49.387860 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:49.387931 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.387933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.388619 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.388620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.389172 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.389174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.389551 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.389551 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.389608 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.390075 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:49.390080 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:49.390364 (+ 284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:49.391915 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:49.391916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:49.391917 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:49.392123 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:49.392379 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:49.392380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:49.392381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:49.392448 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:49.392738 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:49.392740 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:49.392788 (+ 48us) client_negotiation.cc:769] Sending connection context
0924 23:10:49.392854 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 23:10:49.392992 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":63,"threads_started":1}
I20260924 23:10:49.394842 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 23:10:49 dist-test-slave-m0z6 krb5kdc[14203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291449, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 23:10:49 dist-test-slave-m0z6 krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291449, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:49.419085 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.411224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50684 (local address 127.12.212.62:42649)
0924 23:10:49.411315 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:49.411317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:49.411328 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:10:49.411353 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:49.411354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:49.411364 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:49.411428 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:49.412131 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.412673 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.413645 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.413840 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.413902 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.416340 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:49.416342 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:49.416343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:49.416351 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:49.417823 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:49.418220 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:49.418222 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:49.418222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:49.418289 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:49.418556 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:49.418557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:49.418557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:49.418714 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:49.418808 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:49.418934 (+ 126us) server_negotiation.cc:299] Negotiation successful
0924 23:10:49.419006 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 23:10:49.421324 14247 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:10:49.423703 14247 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:10:49.430680 14750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.426636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44354 (local address 127.12.212.2:40019)
0924 23:10:49.427035 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:49.427040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:49.427054 (+ 14us) server_negotiation.cc:407] Connection header received
0924 23:10:49.427111 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:49.427114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:49.427131 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:49.427207 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:10:49.427396 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.428276 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.429712 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.430266 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.430360 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.430437 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:49.430542 (+ 105us) server_negotiation.cc:299] Negotiation successful
0924 23:10:49.430579 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":104,"threads_started":1}
I20260924 23:10:49.430668 14748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.426503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:40019 (local address 127.0.0.1:44354)
0924 23:10:49.426899 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:49.426947 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:49.427027 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:49.427219 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:49.427222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:49.427224 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:10:49.427314 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.427316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.428356 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.428357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.429623 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.429626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.430341 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.430342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.430434 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.430449 (+ 15us) client_negotiation.cc:769] Sending connection context
0924 23:10:49.430517 (+ 68us) client_negotiation.cc:240] Negotiation successful
0924 23:10:49.430577 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":2,"thread_start_us":74,"threads_started":1}
I20260924 23:10:49.430927 14749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.426761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:35911 (local address 127.0.0.1:32852)
0924 23:10:49.426951 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:49.426959 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:49.427016 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:49.427343 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:49.427344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:49.427345 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:10:49.427394 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.427396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.428696 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.428698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.429777 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.429779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.430681 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.430682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.430770 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.430773 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:10:49.430801 (+ 28us) client_negotiation.cc:240] Negotiation successful
0924 23:10:49.430842 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":27,"thread_start_us":71,"threads_started":1}
I20260924 23:10:49.431020 14751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.426730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36060 (local address 127.12.212.3:33329)
0924 23:10:49.426906 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:49.426912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:49.427150 (+ 238us) server_negotiation.cc:407] Connection header received
0924 23:10:49.427191 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:49.427193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:49.427208 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:49.427268 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:10:49.427516 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.428613 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.429697 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.430435 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.430508 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.430564 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:49.430882 (+ 318us) server_negotiation.cc:299] Negotiation successful
0924 23:10:49.430933 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":75,"threads_started":1}
I20260924 23:10:49.431190 14752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.426503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:33329 (local address 127.0.0.1:36060)
0924 23:10:49.426902 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:49.426949 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:49.427025 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:49.427324 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:49.427325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:49.427327 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:10:49.427435 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.427437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.428710 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.428711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.429609 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.429611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.430600 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.430601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.430662 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.430666 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:10:49.430813 (+ 147us) client_negotiation.cc:240] Negotiation successful
0924 23:10:49.430850 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":31,"thread_start_us":105,"threads_started":1}
I20260924 23:10:49.431279 14753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.426832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32852 (local address 127.12.212.1:35911)
0924 23:10:49.427158 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:49.427162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:49.427170 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:10:49.427205 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:49.427207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:49.427222 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:49.427328 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:10:49.427479 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.428613 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.429850 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.430602 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.431054 (+ 452us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.431128 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:49.431152 (+ 24us) server_negotiation.cc:299] Negotiation successful
0924 23:10:49.431193 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":88,"threads_started":1}
I20260924 23:10:49.431762 14665 tablet_service.cc:1467] Processing CreateTablet for tablet d3a1176b30e24e86b6cd416e8e722b40 (DEFAULT_TABLE table=test-table [id=abcee544508b4d648e220d3ecff74203]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:49.431723 14525 tablet_service.cc:1467] Processing CreateTablet for tablet d3a1176b30e24e86b6cd416e8e722b40 (DEFAULT_TABLE table=test-table [id=abcee544508b4d648e220d3ecff74203]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:49.431770 14385 tablet_service.cc:1467] Processing CreateTablet for tablet d3a1176b30e24e86b6cd416e8e722b40 (DEFAULT_TABLE table=test-table [id=abcee544508b4d648e220d3ecff74203]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:49.432092 14665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3a1176b30e24e86b6cd416e8e722b40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:49.432092 14525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3a1176b30e24e86b6cd416e8e722b40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:49.432118 14385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3a1176b30e24e86b6cd416e8e722b40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:49.435319 14755 tablet_bootstrap.cc:492] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a: Bootstrap starting.
I20260924 23:10:49.435418 14754 tablet_bootstrap.cc:492] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970: Bootstrap starting.
I20260924 23:10:49.435647 14756 tablet_bootstrap.cc:492] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc: Bootstrap starting.
I20260924 23:10:49.436019 14754 tablet_bootstrap.cc:654] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:49.436305 14754 log.cc:824] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970: Log is configured to *not* fsync() on all Append() calls
I20260924 23:10:49.436370 14756 tablet_bootstrap.cc:654] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:49.436370 14755 tablet_bootstrap.cc:654] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:49.436690 14756 log.cc:824] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc: Log is configured to *not* fsync() on all Append() calls
I20260924 23:10:49.436690 14755 log.cc:824] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a: Log is configured to *not* fsync() on all Append() calls
I20260924 23:10:49.437040 14754 tablet_bootstrap.cc:492] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970: No bootstrap required, opened a new log
I20260924 23:10:49.437116 14754 ts_tablet_manager.cc:1397] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:10:49.437429 14755 tablet_bootstrap.cc:492] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a: No bootstrap required, opened a new log
I20260924 23:10:49.437503 14755 ts_tablet_manager.cc:1397] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:10:49.437620 14756 tablet_bootstrap.cc:492] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc: No bootstrap required, opened a new log
I20260924 23:10:49.437680 14756 ts_tablet_manager.cc:1397] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:10:49.438959 14754 raft_consensus.cc:348] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.439106 14754 raft_consensus.cc:374] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:49.439137 14754 raft_consensus.cc:729] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc45e7177ecc4d2797873dd4ed416970, State: Initialized, Role: FOLLOWER
I20260924 23:10:49.439263 14754 consensus_queue.cc:260] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.439577 14451 heartbeater.cc:503] Master 127.12.212.62:42649 was elected leader, sending a full tablet report...
I20260924 23:10:49.439595 14755 raft_consensus.cc:348] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.439684 14756 raft_consensus.cc:348] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.439771 14755 raft_consensus.cc:374] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:49.439811 14756 raft_consensus.cc:374] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:49.439810 14755 raft_consensus.cc:729] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e60595e5ce74ff2ae43f557c216d55a, State: Initialized, Role: FOLLOWER
I20260924 23:10:49.439844 14756 raft_consensus.cc:729] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 550ba8a5f1e24c299fd0f708c639abbc, State: Initialized, Role: FOLLOWER
I20260924 23:10:49.439924 14755 consensus_queue.cc:260] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.439972 14756 consensus_queue.cc:260] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.440002 14754 ts_tablet_manager.cc:1428] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:10:49.440215 14756 ts_tablet_manager.cc:1428] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:10:49.440217 14755 ts_tablet_manager.cc:1428] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:10:49.440227 14731 heartbeater.cc:503] Master 127.12.212.62:42649 was elected leader, sending a full tablet report...
I20260924 23:10:49.440322 14591 heartbeater.cc:503] Master 127.12.212.62:42649 was elected leader, sending a full tablet report...
W20260924 23:10:49.474895 14592 tablet.cc:2369] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:10:49.505885 14760 raft_consensus.cc:482] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:10:49.506022 14760 raft_consensus.cc:504] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.506414 14760 leader_election.cc:288] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 550ba8a5f1e24c299fd0f708c639abbc (127.12.212.2:40019), 8e60595e5ce74ff2ae43f557c216d55a (127.12.212.3:33329)
I20260924 23:10:49.509128 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.506605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:40019 (local address 127.12.212.1:41539)
0924 23:10:49.506667 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:49.506679 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:49.506745 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:49.506885 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:49.506886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:49.506887 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:10:49.506955 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.506956 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.507559 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.507560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.508373 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.508376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.508881 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.508882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.508958 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.508962 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:10:49.509013 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 23:10:49.509051 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":31}
I20260924 23:10:49.509126 14750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.506663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:41539 (local address 127.12.212.2:40019)
0924 23:10:49.506730 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:49.506732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:49.506742 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:10:49.506812 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:49.506813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:49.506822 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:49.506870 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:10:49.507011 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.507476 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.508454 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.508812 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.508874 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.508882 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:49.509019 (+ 137us) server_negotiation.cc:299] Negotiation successful
0924 23:10:49.509060 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 23:10:49.509286 14751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.506710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:54173 (local address 127.12.212.3:33329)
0924 23:10:49.506749 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:49.506751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:49.506884 (+ 133us) server_negotiation.cc:407] Connection header received
0924 23:10:49.506940 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:49.506941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:49.506951 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:49.506997 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:10:49.507126 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.507612 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.508572 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.508957 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.509018 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.509026 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:49.509154 (+ 128us) server_negotiation.cc:299] Negotiation successful
0924 23:10:49.509196 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 23:10:49.509442 14763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.506627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:33329 (local address 127.12.212.1:54173)
0924 23:10:49.506811 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:49.506819 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:49.506876 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:49.507003 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:49.507004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:49.507006 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:10:49.507070 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.507072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.507685 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.507686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.508480 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.508482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.509029 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.509030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.509095 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.509098 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:10:49.509142 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 23:10:49.509309 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":125,"threads_started":1}
I20260924 23:10:49.509480 14545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a1176b30e24e86b6cd416e8e722b40" candidate_uuid: "dc45e7177ecc4d2797873dd4ed416970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "550ba8a5f1e24c299fd0f708c639abbc" is_pre_election: true
I20260924 23:10:49.509656 14545 raft_consensus.cc:2454] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc45e7177ecc4d2797873dd4ed416970 in term 0.
I20260924 23:10:49.509792 14685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a1176b30e24e86b6cd416e8e722b40" candidate_uuid: "dc45e7177ecc4d2797873dd4ed416970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e60595e5ce74ff2ae43f557c216d55a" is_pre_election: true
I20260924 23:10:49.509938 14685 raft_consensus.cc:2454] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc45e7177ecc4d2797873dd4ed416970 in term 0.
I20260924 23:10:49.509955 14336 leader_election.cc:302] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 550ba8a5f1e24c299fd0f708c639abbc, dc45e7177ecc4d2797873dd4ed416970; no voters:
I20260924 23:10:49.510074 14760 raft_consensus.cc:2781] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:10:49.510131 14760 raft_consensus.cc:482] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:10:49.510154 14760 raft_consensus.cc:3037] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:49.511164 14760 raft_consensus.cc:504] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.511334 14760 leader_election.cc:288] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [CANDIDATE]: Term 1 election: Requested vote from peers 550ba8a5f1e24c299fd0f708c639abbc (127.12.212.2:40019), 8e60595e5ce74ff2ae43f557c216d55a (127.12.212.3:33329)
I20260924 23:10:49.511507 14685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a1176b30e24e86b6cd416e8e722b40" candidate_uuid: "dc45e7177ecc4d2797873dd4ed416970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e60595e5ce74ff2ae43f557c216d55a"
I20260924 23:10:49.511565 14545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a1176b30e24e86b6cd416e8e722b40" candidate_uuid: "dc45e7177ecc4d2797873dd4ed416970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "550ba8a5f1e24c299fd0f708c639abbc"
I20260924 23:10:49.511601 14685 raft_consensus.cc:3037] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:49.511631 14545 raft_consensus.cc:3037] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:49.512326 14545 raft_consensus.cc:2454] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc45e7177ecc4d2797873dd4ed416970 in term 1.
I20260924 23:10:49.512331 14685 raft_consensus.cc:2454] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc45e7177ecc4d2797873dd4ed416970 in term 1.
I20260924 23:10:49.512511 14338 leader_election.cc:302] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e60595e5ce74ff2ae43f557c216d55a, dc45e7177ecc4d2797873dd4ed416970; no voters:
I20260924 23:10:49.512602 14760 raft_consensus.cc:2781] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:10:49.512763 14760 raft_consensus.cc:686] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [term 1 LEADER]: Becoming Leader. State: Replica: dc45e7177ecc4d2797873dd4ed416970, State: Running, Role: LEADER
I20260924 23:10:49.512861 14760 consensus_queue.cc:237] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.513844 14247 catalog_manager.cc:5495] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc45e7177ecc4d2797873dd4ed416970 (127.12.212.1). New cstate: current_term: 1 leader_uuid: "dc45e7177ecc4d2797873dd4ed416970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20260924 23:10:49.525183 14753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.522675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32858 (local address 127.12.212.1:35911)
0924 23:10:49.522774 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:49.522776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:49.522784 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:10:49.522831 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:49.522832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:49.522847 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:49.522901 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:10:49.523041 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.523534 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.524323 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.524557 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.524615 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.524811 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:10:49.524928 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:10:49.524999 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:49.525057 (+ 58us) server_negotiation.cc:299] Negotiation successful
0924 23:10:49.525081 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260924 23:10:49.527053 14685 raft_consensus.cc:1261] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a [term 1 FOLLOWER]: Refusing update from remote peer dc45e7177ecc4d2797873dd4ed416970: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:10:49.527134 14545 raft_consensus.cc:1261] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc [term 1 FOLLOWER]: Refusing update from remote peer dc45e7177ecc4d2797873dd4ed416970: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:10:49.527392 14764 consensus_queue.cc:1035] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:10:49.527535 14760 consensus_queue.cc:1035] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:10:49.529819 14769 mvcc.cc:205] Tried to move back new op lower bound from 7333033777260158976 to 7333033777205649408. Current Snapshot: MvccSnapshot[applied={T|T < 7333033777260158976}]
I20260924 23:10:49.533699 14247 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:10:49.533790 14247 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:10:49.534530 14247 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c: Sending DeleteTablet for 3 replicas of tablet d3a1176b30e24e86b6cd416e8e722b40
I20260924 23:10:49.534914 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet d3a1176b30e24e86b6cd416e8e722b40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:10:49 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:36060
I20260924 23:10:49.534919 14525 tablet_service.cc:1514] Processing DeleteTablet for tablet d3a1176b30e24e86b6cd416e8e722b40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:10:49 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:44354
I20260924 23:10:49.535261 14385 tablet_service.cc:1514] Processing DeleteTablet for tablet d3a1176b30e24e86b6cd416e8e722b40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:10:49 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:32852
I20260924 23:10:49.535212 14247 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:10:49.535365 14247 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:10:49.535468 14778 tablet_replica.cc:330] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970: stopping tablet replica
I20260924 23:10:49.535557 14778 raft_consensus.cc:2229] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:10:49.535665 14778 raft_consensus.cc:2258] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:10:49.536149 14778 ts_tablet_manager.cc:1905] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:10:49.536755 14385 tablet_service.cc:1467] Processing CreateTablet for tablet c6f9775aaabe459bac72ac5c2d85fbb0 (DEFAULT_TABLE table=test-table [id=595c66514e8840b4aa99be23dba07a94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:49.536811 14665 tablet_service.cc:1467] Processing CreateTablet for tablet c6f9775aaabe459bac72ac5c2d85fbb0 (DEFAULT_TABLE table=test-table [id=595c66514e8840b4aa99be23dba07a94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:49.536872 14385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6f9775aaabe459bac72ac5c2d85fbb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:49.536947 14665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6f9775aaabe459bac72ac5c2d85fbb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:49.537072 14776 tablet_replica.cc:330] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc: stopping tablet replica
I20260924 23:10:49.537199 14776 raft_consensus.cc:2229] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:10:49.537304 14776 raft_consensus.cc:2258] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:10:49.537474 14777 tablet_replica.cc:330] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a: stopping tablet replica
I20260924 23:10:49.537577 14777 raft_consensus.cc:2229] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:10:49.537907 14777 raft_consensus.cc:2258] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:10:49.538395 14754 tablet_bootstrap.cc:492] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970: Bootstrap starting.
I20260924 23:10:49.538463 14755 tablet_bootstrap.cc:492] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a: Bootstrap starting.
I20260924 23:10:49.538472 14525 tablet_service.cc:1467] Processing CreateTablet for tablet c6f9775aaabe459bac72ac5c2d85fbb0 (DEFAULT_TABLE table=test-table [id=595c66514e8840b4aa99be23dba07a94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:49.538573 14525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6f9775aaabe459bac72ac5c2d85fbb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:49.538829 14754 tablet_bootstrap.cc:654] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:49.538878 14755 tablet_bootstrap.cc:654] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:49.539539 14755 tablet_bootstrap.cc:492] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a: No bootstrap required, opened a new log
I20260924 23:10:49.539589 14755 ts_tablet_manager.cc:1397] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 23:10:49.539717 14755 raft_consensus.cc:348] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.539803 14755 raft_consensus.cc:374] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:49.539824 14755 raft_consensus.cc:729] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e60595e5ce74ff2ae43f557c216d55a, State: Initialized, Role: FOLLOWER
I20260924 23:10:49.539872 14755 consensus_queue.cc:260] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.539903 14756 tablet_bootstrap.cc:492] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc: Bootstrap starting.
I20260924 23:10:49.539961 14755 ts_tablet_manager.cc:1428] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 23:10:49.540302 14756 tablet_bootstrap.cc:654] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:49.540891 14776 ts_tablet_manager.cc:1905] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:10:49.542122 14778 ts_tablet_manager.cc:1918] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:10:49.542207 14778 log.cc:1196] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-0/wal/wals/d3a1176b30e24e86b6cd416e8e722b40
I20260924 23:10:49.542447 14776 ts_tablet_manager.cc:1918] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:10:49.542484 14778 ts_tablet_manager.cc:1939] T d3a1176b30e24e86b6cd416e8e722b40 P dc45e7177ecc4d2797873dd4ed416970: Deleting consensus metadata
I20260924 23:10:49.542624 14776 log.cc:1196] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/d3a1176b30e24e86b6cd416e8e722b40
I20260924 23:10:49.542954 14776 ts_tablet_manager.cc:1939] T d3a1176b30e24e86b6cd416e8e722b40 P 550ba8a5f1e24c299fd0f708c639abbc: Deleting consensus metadata
I20260924 23:10:49.542939 14232 catalog_manager.cc:4841] TS dc45e7177ecc4d2797873dd4ed416970 (127.12.212.1:35911): tablet d3a1176b30e24e86b6cd416e8e722b40 (table test-table [id=abcee544508b4d648e220d3ecff74203]) successfully deleted
I20260924 23:10:49.542969 14777 ts_tablet_manager.cc:1905] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:10:49.543363 14754 tablet_bootstrap.cc:492] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970: No bootstrap required, opened a new log
I20260924 23:10:49.543416 14754 ts_tablet_manager.cc:1397] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260924 23:10:49.543507 14232 catalog_manager.cc:4841] TS 550ba8a5f1e24c299fd0f708c639abbc (127.12.212.2:40019): tablet d3a1176b30e24e86b6cd416e8e722b40 (table test-table [id=abcee544508b4d648e220d3ecff74203]) successfully deleted
I20260924 23:10:49.543581 14754 raft_consensus.cc:348] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.543663 14754 raft_consensus.cc:374] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:49.543682 14754 raft_consensus.cc:729] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc45e7177ecc4d2797873dd4ed416970, State: Initialized, Role: FOLLOWER
I20260924 23:10:49.543725 14754 consensus_queue.cc:260] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.543809 14754 ts_tablet_manager.cc:1428] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 23:10:49.544561 14777 ts_tablet_manager.cc:1918] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:10:49.544689 14777 log.cc:1196] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/d3a1176b30e24e86b6cd416e8e722b40
I20260924 23:10:49.544763 14756 tablet_bootstrap.cc:492] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc: No bootstrap required, opened a new log
I20260924 23:10:49.544833 14756 ts_tablet_manager.cc:1397] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260924 23:10:49.544991 14756 raft_consensus.cc:348] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.545063 14756 raft_consensus.cc:374] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:49.545081 14756 raft_consensus.cc:729] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 550ba8a5f1e24c299fd0f708c639abbc, State: Initialized, Role: FOLLOWER
I20260924 23:10:49.545120 14756 consensus_queue.cc:260] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.545193 14756 ts_tablet_manager.cc:1428] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 23:10:49.545563 14777 ts_tablet_manager.cc:1939] T d3a1176b30e24e86b6cd416e8e722b40 P 8e60595e5ce74ff2ae43f557c216d55a: Deleting consensus metadata
I20260924 23:10:49.546015 14234 catalog_manager.cc:4841] TS 8e60595e5ce74ff2ae43f557c216d55a (127.12.212.3:33329): tablet d3a1176b30e24e86b6cd416e8e722b40 (table test-table [id=abcee544508b4d648e220d3ecff74203]) successfully deleted
W20260924 23:10:49.567533 14452 tablet.cc:2369] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:10:49.568195 14764 raft_consensus.cc:482] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:10:49.568291 14764 raft_consensus.cc:504] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.568461 14764 leader_election.cc:288] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 550ba8a5f1e24c299fd0f708c639abbc (127.12.212.2:40019), 8e60595e5ce74ff2ae43f557c216d55a (127.12.212.3:33329)
I20260924 23:10:49.568678 14685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6f9775aaabe459bac72ac5c2d85fbb0" candidate_uuid: "dc45e7177ecc4d2797873dd4ed416970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e60595e5ce74ff2ae43f557c216d55a" is_pre_election: true
I20260924 23:10:49.568678 14545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6f9775aaabe459bac72ac5c2d85fbb0" candidate_uuid: "dc45e7177ecc4d2797873dd4ed416970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "550ba8a5f1e24c299fd0f708c639abbc" is_pre_election: true
I20260924 23:10:49.568789 14685 raft_consensus.cc:2454] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc45e7177ecc4d2797873dd4ed416970 in term 0.
I20260924 23:10:49.568789 14545 raft_consensus.cc:2454] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc45e7177ecc4d2797873dd4ed416970 in term 0.
I20260924 23:10:49.568943 14338 leader_election.cc:302] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e60595e5ce74ff2ae43f557c216d55a, dc45e7177ecc4d2797873dd4ed416970; no voters:
I20260924 23:10:49.569058 14764 raft_consensus.cc:2781] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:10:49.569092 14764 raft_consensus.cc:482] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:10:49.569106 14764 raft_consensus.cc:3037] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:49.569825 14764 raft_consensus.cc:504] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.569957 14764 leader_election.cc:288] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [CANDIDATE]: Term 1 election: Requested vote from peers 550ba8a5f1e24c299fd0f708c639abbc (127.12.212.2:40019), 8e60595e5ce74ff2ae43f557c216d55a (127.12.212.3:33329)
I20260924 23:10:49.570147 14685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6f9775aaabe459bac72ac5c2d85fbb0" candidate_uuid: "dc45e7177ecc4d2797873dd4ed416970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e60595e5ce74ff2ae43f557c216d55a"
I20260924 23:10:49.570151 14545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6f9775aaabe459bac72ac5c2d85fbb0" candidate_uuid: "dc45e7177ecc4d2797873dd4ed416970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "550ba8a5f1e24c299fd0f708c639abbc"
I20260924 23:10:49.570230 14685 raft_consensus.cc:3037] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:49.570230 14545 raft_consensus.cc:3037] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:49.571003 14685 raft_consensus.cc:2454] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc45e7177ecc4d2797873dd4ed416970 in term 1.
I20260924 23:10:49.571003 14545 raft_consensus.cc:2454] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc45e7177ecc4d2797873dd4ed416970 in term 1.
I20260924 23:10:49.571190 14336 leader_election.cc:302] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 550ba8a5f1e24c299fd0f708c639abbc, dc45e7177ecc4d2797873dd4ed416970; no voters:
I20260924 23:10:49.571308 14764 raft_consensus.cc:2781] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:10:49.571374 14764 raft_consensus.cc:686] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [term 1 LEADER]: Becoming Leader. State: Replica: dc45e7177ecc4d2797873dd4ed416970, State: Running, Role: LEADER
I20260924 23:10:49.571434 14764 consensus_queue.cc:237] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.571990 14248 catalog_manager.cc:5495] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc45e7177ecc4d2797873dd4ed416970 (127.12.212.1). New cstate: current_term: 1 leader_uuid: "dc45e7177ecc4d2797873dd4ed416970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } health_report { overall_health: UNKNOWN } } }
Sep 24 23:10:49 dist-test-slave-m0z6 krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291448, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:49.598372 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.591611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50694 (local address 127.12.212.62:42649)
0924 23:10:49.591672 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:49.591675 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:49.591802 (+ 127us) server_negotiation.cc:407] Connection header received
0924 23:10:49.591926 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:49.591927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:49.591936 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:49.591982 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:49.592469 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.592950 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.593605 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.593778 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.593831 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.595992 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:49.595993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:49.595994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:49.596001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:49.597198 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:49.597652 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:49.597653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:49.597653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:49.597701 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:49.597909 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:49.597909 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:49.597909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:49.598012 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:49.598056 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:49.598230 (+ 174us) server_negotiation.cc:299] Negotiation successful
0924 23:10:49.598301 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":35}
I20260924 23:10:49.598490 14788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.591550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42649 (local address 127.0.0.1:50694)
0924 23:10:49.591733 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:49.591742 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:49.591806 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:49.591992 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:49.591993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:49.592329 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:49.592392 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.592395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.593019 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.593019 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.593535 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.593537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.593834 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.593834 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.593882 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.594400 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:49.594405 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:49.595888 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:49.597283 (+ 1395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:49.597291 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:49.597294 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:49.597581 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:49.597764 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:49.597765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:49.597766 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:49.597843 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:49.598069 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:49.598077 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:49.598136 (+ 59us) client_negotiation.cc:769] Sending connection context
0924 23:10:49.598216 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 23:10:49.598376 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":35,"thread_start_us":81,"threads_started":1}
I20260924 23:10:49.600564 14248 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:50694:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 23:10:49.600771 14248 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:10:49.602115 14385 tablet_service.cc:1467] Processing CreateTablet for tablet 89985ccf28a6422fb1bbbda8f1b2a3ae (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=81e46ea0d90a455487ecf2e1c26c96b4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 23:10:49.602128 14525 tablet_service.cc:1467] Processing CreateTablet for tablet 89985ccf28a6422fb1bbbda8f1b2a3ae (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=81e46ea0d90a455487ecf2e1c26c96b4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 23:10:49.602275 14525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89985ccf28a6422fb1bbbda8f1b2a3ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:49.602276 14385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89985ccf28a6422fb1bbbda8f1b2a3ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:49.602236 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 89985ccf28a6422fb1bbbda8f1b2a3ae (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=81e46ea0d90a455487ecf2e1c26c96b4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 23:10:49.602375 14665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89985ccf28a6422fb1bbbda8f1b2a3ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:49.603804 14754 tablet_bootstrap.cc:492] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970: Bootstrap starting.
I20260924 23:10:49.603794 14756 tablet_bootstrap.cc:492] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 550ba8a5f1e24c299fd0f708c639abbc: Bootstrap starting.
I20260924 23:10:49.604198 14754 tablet_bootstrap.cc:654] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:49.604200 14756 tablet_bootstrap.cc:654] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 550ba8a5f1e24c299fd0f708c639abbc: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:49.604532 14755 tablet_bootstrap.cc:492] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a: Bootstrap starting.
I20260924 23:10:49.604900 14754 tablet_bootstrap.cc:492] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970: No bootstrap required, opened a new log
I20260924 23:10:49.604944 14754 ts_tablet_manager.cc:1397] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 23:10:49.604956 14755 tablet_bootstrap.cc:654] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:49.605095 14754 raft_consensus.cc:348] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.605165 14754 raft_consensus.cc:374] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:49.605183 14754 raft_consensus.cc:729] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc45e7177ecc4d2797873dd4ed416970, State: Initialized, Role: FOLLOWER
I20260924 23:10:49.605450 14756 tablet_bootstrap.cc:492] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 550ba8a5f1e24c299fd0f708c639abbc: No bootstrap required, opened a new log
I20260924 23:10:49.605494 14756 ts_tablet_manager.cc:1397] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 550ba8a5f1e24c299fd0f708c639abbc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:10:49.605639 14756 raft_consensus.cc:348] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 550ba8a5f1e24c299fd0f708c639abbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.605726 14756 raft_consensus.cc:374] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 550ba8a5f1e24c299fd0f708c639abbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:49.605746 14756 raft_consensus.cc:729] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 550ba8a5f1e24c299fd0f708c639abbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 550ba8a5f1e24c299fd0f708c639abbc, State: Initialized, Role: FOLLOWER
I20260924 23:10:49.605773 14755 tablet_bootstrap.cc:492] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a: No bootstrap required, opened a new log
I20260924 23:10:49.605782 14756 consensus_queue.cc:260] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 550ba8a5f1e24c299fd0f708c639abbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.605818 14755 ts_tablet_manager.cc:1397] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 23:10:49.605885 14756 ts_tablet_manager.cc:1428] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 550ba8a5f1e24c299fd0f708c639abbc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 23:10:49.605950 14755 raft_consensus.cc:348] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.606017 14755 raft_consensus.cc:374] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:49.606036 14755 raft_consensus.cc:729] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e60595e5ce74ff2ae43f557c216d55a, State: Initialized, Role: FOLLOWER
I20260924 23:10:49.606014 14762 tablet_replica.cc:439] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 550ba8a5f1e24c299fd0f708c639abbc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } } }
I20260924 23:10:49.606086 14762 tablet_replica.cc:442] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 550ba8a5f1e24c299fd0f708c639abbc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:10:49.606081 14755 consensus_queue.cc:260] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.606159 14755 ts_tablet_manager.cc:1428] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 23:10:49.605212 14754 consensus_queue.cc:260] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.606318 14761 tablet_replica.cc:439] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } } }
I20260924 23:10:49.606390 14754 ts_tablet_manager.cc:1428] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260924 23:10:49.606395 14761 tablet_replica.cc:442] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:10:49.606472 14764 tablet_replica.cc:439] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } } }
I20260924 23:10:49.606528 14764 tablet_replica.cc:442] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970: This TxnStatusTablet replica's current role is: FOLLOWER
W20260924 23:10:49.632889 14732 tablet.cc:2369] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:10:49.933174 14764 raft_consensus.cc:482] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:10:49.933272 14764 raft_consensus.cc:504] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.933421 14764 leader_election.cc:288] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 550ba8a5f1e24c299fd0f708c639abbc (127.12.212.2:40019), 8e60595e5ce74ff2ae43f557c216d55a (127.12.212.3:33329)
I20260924 23:10:49.933653 14685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89985ccf28a6422fb1bbbda8f1b2a3ae" candidate_uuid: "dc45e7177ecc4d2797873dd4ed416970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e60595e5ce74ff2ae43f557c216d55a" is_pre_election: true
I20260924 23:10:49.933653 14545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89985ccf28a6422fb1bbbda8f1b2a3ae" candidate_uuid: "dc45e7177ecc4d2797873dd4ed416970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "550ba8a5f1e24c299fd0f708c639abbc" is_pre_election: true
I20260924 23:10:49.933756 14685 raft_consensus.cc:2454] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc45e7177ecc4d2797873dd4ed416970 in term 0.
I20260924 23:10:49.933756 14545 raft_consensus.cc:2454] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 550ba8a5f1e24c299fd0f708c639abbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc45e7177ecc4d2797873dd4ed416970 in term 0.
I20260924 23:10:49.933923 14338 leader_election.cc:302] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e60595e5ce74ff2ae43f557c216d55a, dc45e7177ecc4d2797873dd4ed416970; no voters:
I20260924 23:10:49.934032 14764 raft_consensus.cc:2781] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:10:49.934092 14764 raft_consensus.cc:482] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:10:49.934106 14764 raft_consensus.cc:3037] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:49.934829 14764 raft_consensus.cc:504] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.934949 14764 leader_election.cc:288] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 [CANDIDATE]: Term 1 election: Requested vote from peers 550ba8a5f1e24c299fd0f708c639abbc (127.12.212.2:40019), 8e60595e5ce74ff2ae43f557c216d55a (127.12.212.3:33329)
I20260924 23:10:49.935104 14545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89985ccf28a6422fb1bbbda8f1b2a3ae" candidate_uuid: "dc45e7177ecc4d2797873dd4ed416970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "550ba8a5f1e24c299fd0f708c639abbc"
I20260924 23:10:49.935128 14685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89985ccf28a6422fb1bbbda8f1b2a3ae" candidate_uuid: "dc45e7177ecc4d2797873dd4ed416970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e60595e5ce74ff2ae43f557c216d55a"
I20260924 23:10:49.935191 14685 raft_consensus.cc:3037] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:49.935191 14545 raft_consensus.cc:3037] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 550ba8a5f1e24c299fd0f708c639abbc [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:49.935915 14545 raft_consensus.cc:2454] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 550ba8a5f1e24c299fd0f708c639abbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc45e7177ecc4d2797873dd4ed416970 in term 1.
I20260924 23:10:49.935915 14685 raft_consensus.cc:2454] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc45e7177ecc4d2797873dd4ed416970 in term 1.
I20260924 23:10:49.936139 14336 leader_election.cc:302] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 550ba8a5f1e24c299fd0f708c639abbc, dc45e7177ecc4d2797873dd4ed416970; no voters:
I20260924 23:10:49.936234 14764 raft_consensus.cc:2781] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:10:49.936282 14764 raft_consensus.cc:686] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 [term 1 LEADER]: Becoming Leader. State: Replica: dc45e7177ecc4d2797873dd4ed416970, State: Running, Role: LEADER
I20260924 23:10:49.936322 14764 consensus_queue.cc:237] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } }
I20260924 23:10:49.936566 14760 tablet_replica.cc:439] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970: TxnStatusTablet state changed. Reason: New leader dc45e7177ecc4d2797873dd4ed416970. Latest consensus state: current_term: 1 leader_uuid: "dc45e7177ecc4d2797873dd4ed416970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } } }
I20260924 23:10:49.936638 14760 tablet_replica.cc:442] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970: This TxnStatusTablet replica's current role is: LEADER
I20260924 23:10:49.936777 14793 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 23:10:49.936884 14793 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 23:10:49.936908 14247 catalog_manager.cc:5495] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc45e7177ecc4d2797873dd4ed416970 (127.12.212.1). New cstate: current_term: 1 leader_uuid: "dc45e7177ecc4d2797873dd4ed416970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20260924 23:10:49.963929 14788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.961263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:35911 (local address 127.0.0.1:32870)
0924 23:10:49.961334 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:49.961346 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:49.961393 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:49.961537 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:49.961537 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:49.961538 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 23:10:49.961587 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.961589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.962429 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.962430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.963085 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.963087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.963545 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.963546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.963597 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.963633 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 23:10:49.963792 (+ 159us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 23:10:49.963796 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:10:49.963823 (+ 27us) client_negotiation.cc:240] Negotiation successful
0924 23:10:49.963868 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 23:10:49.963932 14753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.961357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32870 (local address 127.12.212.1:35911)
0924 23:10:49.961411 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:49.961414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:49.961426 (+ 12us) server_negotiation.cc:407] Connection header received
0924 23:10:49.961453 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:49.961454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:49.961467 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:49.961532 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:10:49.961626 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.962363 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.963154 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.963475 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.963534 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.963706 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:10:49.963741 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:10:49.963791 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:49.963828 (+ 37us) server_negotiation.cc:299] Negotiation successful
0924 23:10:49.963851 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 23:10:49.964983 14545 raft_consensus.cc:1261] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 550ba8a5f1e24c299fd0f708c639abbc [term 1 FOLLOWER]: Refusing update from remote peer dc45e7177ecc4d2797873dd4ed416970: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:10:49.965019 14685 raft_consensus.cc:1261] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a [term 1 FOLLOWER]: Refusing update from remote peer dc45e7177ecc4d2797873dd4ed416970: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:10:49.965189 14764 consensus_queue.cc:1035] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:10:49.965307 14760 consensus_queue.cc:1035] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:10:49.965730 14762 tablet_replica.cc:439] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 550ba8a5f1e24c299fd0f708c639abbc: TxnStatusTablet state changed. Reason: New leader dc45e7177ecc4d2797873dd4ed416970. Latest consensus state: current_term: 1 leader_uuid: "dc45e7177ecc4d2797873dd4ed416970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } } }
I20260924 23:10:49.965822 14762 tablet_replica.cc:442] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 550ba8a5f1e24c299fd0f708c639abbc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:10:49.965797 14761 tablet_replica.cc:439] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a: TxnStatusTablet state changed. Reason: New leader dc45e7177ecc4d2797873dd4ed416970. Latest consensus state: current_term: 1 leader_uuid: "dc45e7177ecc4d2797873dd4ed416970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } } }
I20260924 23:10:49.965868 14761 tablet_replica.cc:442] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:10:49.966423 14760 tablet_replica.cc:439] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dc45e7177ecc4d2797873dd4ed416970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } } }
I20260924 23:10:49.966490 14760 tablet_replica.cc:442] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970: This TxnStatusTablet replica's current role is: LEADER
I20260924 23:10:49.966455 14767 mvcc.cc:205] Tried to move back new op lower bound from 7333033779055030272 to 7333033778939691008. Current Snapshot: MvccSnapshot[applied={T|T < 7333033779055030272}]
I20260924 23:10:49.966555 14760 tablet_replica.cc:439] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dc45e7177ecc4d2797873dd4ed416970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } } }
I20260924 23:10:49.966593 14760 tablet_replica.cc:442] T 89985ccf28a6422fb1bbbda8f1b2a3ae P dc45e7177ecc4d2797873dd4ed416970: This TxnStatusTablet replica's current role is: LEADER
I20260924 23:10:49.966910 14761 tablet_replica.cc:439] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dc45e7177ecc4d2797873dd4ed416970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } } }
I20260924 23:10:49.966989 14761 tablet_replica.cc:442] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 8e60595e5ce74ff2ae43f557c216d55a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:10:49.967229 14762 tablet_replica.cc:439] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 550ba8a5f1e24c299fd0f708c639abbc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dc45e7177ecc4d2797873dd4ed416970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc45e7177ecc4d2797873dd4ed416970" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 35911 } } peers { permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 } } peers { permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 } } }
I20260924 23:10:49.967311 14762 tablet_replica.cc:442] T 89985ccf28a6422fb1bbbda8f1b2a3ae P 550ba8a5f1e24c299fd0f708c639abbc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:10:49.973325 14753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.970462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:56573 (local address 127.12.212.1:35911)
0924 23:10:49.970495 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:49.970497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:49.970538 (+ 41us) server_negotiation.cc:407] Connection header received
0924 23:10:49.970589 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:49.970589 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:49.970598 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:49.970655 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:10:49.970824 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.971614 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.972534 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.972826 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.972891 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.973031 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:10:49.973064 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:10:49.973109 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:49.973221 (+ 112us) server_negotiation.cc:299] Negotiation successful
0924 23:10:49.973240 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 23:10:49.973325 14800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.970340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:35911 (local address 127.12.212.1:56573)
0924 23:10:49.970480 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:49.970491 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:49.970549 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:49.970659 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:49.970659 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:49.970669 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 23:10:49.970763 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.970765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.971682 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.971683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.972465 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:49.972467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.972899 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.972900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:49.972944 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:49.972959 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 23:10:49.973159 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 23:10:49.973162 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:10:49.973214 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 23:10:49.973255 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260924 23:10:49.975773 14685 raft_consensus.cc:1261] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a [term 1 FOLLOWER]: Refusing update from remote peer dc45e7177ecc4d2797873dd4ed416970: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:10:49.975775 14545 raft_consensus.cc:1261] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc [term 1 FOLLOWER]: Refusing update from remote peer dc45e7177ecc4d2797873dd4ed416970: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:10:49.975984 14801 consensus_queue.cc:1035] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "550ba8a5f1e24c299fd0f708c639abbc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:10:49.976112 14764 consensus_queue.cc:1035] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e60595e5ce74ff2ae43f557c216d55a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:10:49.990228 14247 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:10:49.990345 14247 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:10:49.991032 14247 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52a022b9db364b9384ddceb76007e38c: Sending DeleteTablet for 3 replicas of tablet c6f9775aaabe459bac72ac5c2d85fbb0
I20260924 23:10:49.991333 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet c6f9775aaabe459bac72ac5c2d85fbb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:10:49 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:36060
I20260924 23:10:49.991461 14777 tablet_replica.cc:330] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a: stopping tablet replica
I20260924 23:10:49.991447 14384 tablet_service.cc:1514] Processing DeleteTablet for tablet c6f9775aaabe459bac72ac5c2d85fbb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:10:49 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:32852
I20260924 23:10:49.991513 14777 raft_consensus.cc:2229] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:10:49.991528 14778 tablet_replica.cc:330] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970: stopping tablet replica
I20260924 23:10:49.991551 14777 raft_consensus.cc:2258] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:10:49.991564 14778 raft_consensus.cc:2229] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:10:49.991619 14778 raft_consensus.cc:2258] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:10:49.991751 14525 tablet_service.cc:1514] Processing DeleteTablet for tablet c6f9775aaabe459bac72ac5c2d85fbb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:10:49 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:44354
I20260924 23:10:49.991859 14776 tablet_replica.cc:330] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc: stopping tablet replica
I20260924 23:10:49.991932 14776 raft_consensus.cc:2229] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:10:49.991950 14778 ts_tablet_manager.cc:1905] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:10:49.991981 14776 raft_consensus.cc:2258] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:10:49.992275 14777 ts_tablet_manager.cc:1905] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:10:49.993577 14778 ts_tablet_manager.cc:1918] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 23:10:49.993630 14778 log.cc:1196] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-0/wal/wals/c6f9775aaabe459bac72ac5c2d85fbb0
I20260924 23:10:49.993667 14777 ts_tablet_manager.cc:1918] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 23:10:49.993711 14777 log.cc:1196] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/c6f9775aaabe459bac72ac5c2d85fbb0
I20260924 23:10:49.993826 14778 ts_tablet_manager.cc:1939] T c6f9775aaabe459bac72ac5c2d85fbb0 P dc45e7177ecc4d2797873dd4ed416970: Deleting consensus metadata
I20260924 23:10:49.993888 14777 ts_tablet_manager.cc:1939] T c6f9775aaabe459bac72ac5c2d85fbb0 P 8e60595e5ce74ff2ae43f557c216d55a: Deleting consensus metadata
I20260924 23:10:49.994354 14232 catalog_manager.cc:4841] TS dc45e7177ecc4d2797873dd4ed416970 (127.12.212.1:35911): tablet c6f9775aaabe459bac72ac5c2d85fbb0 (table test-table [id=595c66514e8840b4aa99be23dba07a94]) successfully deleted
I20260924 23:10:49.994398 14234 catalog_manager.cc:4841] TS 8e60595e5ce74ff2ae43f557c216d55a (127.12.212.3:33329): tablet c6f9775aaabe459bac72ac5c2d85fbb0 (table test-table [id=595c66514e8840b4aa99be23dba07a94]) successfully deleted
I20260924 23:10:49.994658 14776 ts_tablet_manager.cc:1905] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:10:49.996480 14776 ts_tablet_manager.cc:1918] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 23:10:49.996549 14776 log.cc:1196] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/c6f9775aaabe459bac72ac5c2d85fbb0
I20260924 23:10:49.996743 14776 ts_tablet_manager.cc:1939] T c6f9775aaabe459bac72ac5c2d85fbb0 P 550ba8a5f1e24c299fd0f708c639abbc: Deleting consensus metadata
I20260924 23:10:49.997295 14232 catalog_manager.cc:4841] TS 550ba8a5f1e24c299fd0f708c639abbc (127.12.212.2:40019): tablet c6f9775aaabe459bac72ac5c2d85fbb0 (table test-table [id=595c66514e8840b4aa99be23dba07a94]) successfully deleted
Sep 24 23:10:49 dist-test-slave-m0z6 krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291449, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.212.1@KRBTEST.COM
I20260924 23:10:50.004057 14753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:49.994054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32884 (local address 127.12.212.1:35911)
0924 23:10:49.994097 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:49.994099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:49.994721 (+ 622us) server_negotiation.cc:407] Connection header received
0924 23:10:49.994805 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:49.994806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:49.994818 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:49.994849 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:49.995801 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.996555 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.997577 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:49.997776 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:49.997836 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:50.000908 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:50.000924 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:50.000925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:50.000934 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:50.002479 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:50.002930 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:50.002934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:50.002935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:50.003008 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:50.003247 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:50.003248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:50.003249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:50.003493 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:50.003668 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:50.003849 (+ 181us) server_negotiation.cc:299] Negotiation successful
0924 23:10:50.003953 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260924 23:10:50.004542 14445 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32884
I20260924 23:10:50.011835 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:50.005570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50708 (local address 127.12.212.62:42649)
0924 23:10:50.005634 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:50.005636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:50.005644 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:10:50.005669 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:50.005670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:50.005679 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:50.005725 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:50.006273 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:50.006866 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:50.007661 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:50.007873 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:50.007931 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:50.009330 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:50.009331 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:50.009332 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:50.009341 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:50.010808 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:50.011116 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:50.011118 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:50.011118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:50.011178 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:50.011337 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:50.011338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:50.011338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:50.011453 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:50.011541 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:50.011661 (+ 120us) server_negotiation.cc:299] Negotiation successful
0924 23:10:50.011738 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260924 23:10:50.012068 14247 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50708
I20260924 23:10:50.012652 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 14320
I20260924 23:10:50.018889 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 14460
I20260924 23:10:50.022984 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 14600
I20260924 23:10:50.027236 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 14216
2026-09-24T23:10:50Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2337 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:10:50 dist-test-slave-m0z6 krb5kdc[14811](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:10:50 dist-test-slave-m0z6 krb5kdc[14811](info): set up 2 sockets
Sep 24 23:10:50 dist-test-slave-m0z6 krb5kdc[14811](info): commencing operation
krb5kdc: starting...
W20260924 23:10:50.978327 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:10:51 dist-test-slave-m0z6 krb5kdc[14811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:10:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:10:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:10:51.097294 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:39035
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:39035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:51.176959 14824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:51.177196 14824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:51.177220 14824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:51.178433 14824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:10:51.178479 14824 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 23:10:51.178493 14824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:51.178503 14824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:51.178514 14824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:10:51.178524 14824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:10:51.180078 14824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:39035
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:39035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:51.180302 14824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:51.180485 14824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:51.185555 14832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:51.185556 14830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:51.185598 14824 server_base.cc:1034] running on GCE node
W20260924 23:10:51.185556 14829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:51.186043 14824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:51.186343 14824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:51.187490 14824 hybrid_clock.cc:648] HybridClock initialized: now 1790291451187474 us; error 35 us; skew 500 ppm
Sep 24 23:10:51 dist-test-slave-m0z6 krb5kdc[14811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291451, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:51.190116 14824 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:10:51.190618 14824 webserver.cc:460] Webserver started at http://127.12.212.62:44793/ using document root <none> and password file <none>
I20260924 23:10:51.190856 14824 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:51.190907 14824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:51.191036 14824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:51.192131 14824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "40524990b4494484920a2b5f044b3a71"
format_stamp: "Formatted at 2026-09-24 23:10:51 on dist-test-slave-m0z6"
server_key: "511502976bd6b209e33a9b0aed76927f"
server_key_iv: "b92577e76f28e21b42ddeb54d00fefa4"
server_key_version: "encryptionkey@0"
I20260924 23:10:51.192487 14824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "40524990b4494484920a2b5f044b3a71"
format_stamp: "Formatted at 2026-09-24 23:10:51 on dist-test-slave-m0z6"
server_key: "511502976bd6b209e33a9b0aed76927f"
server_key_iv: "b92577e76f28e21b42ddeb54d00fefa4"
server_key_version: "encryptionkey@0"
I20260924 23:10:51.193919 14824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:10:51.194779 14839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:51.194957 14824 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:10:51.195044 14824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "40524990b4494484920a2b5f044b3a71"
format_stamp: "Formatted at 2026-09-24 23:10:51 on dist-test-slave-m0z6"
server_key: "511502976bd6b209e33a9b0aed76927f"
server_key_iv: "b92577e76f28e21b42ddeb54d00fefa4"
server_key_version: "encryptionkey@0"
I20260924 23:10:51.195117 14824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:51.214658 14824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:51.216863 14824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:51.217011 14824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:51.221498 14824 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:39035
I20260924 23:10:51.221556 14901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:39035 every 8 connection(s)
I20260924 23:10:51.221877 14824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:10:51.222325 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 14824
I20260924 23:10:51.222426 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:10:51.222602 13136 external_mini_cluster.cc:1428] Setting key 7b3f28bd41fc9823c910b120c75cb855
I20260924 23:10:51.222616 14902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:51.225462 14902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71: Bootstrap starting.
I20260924 23:10:51.226030 14902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:51.226289 14902 log.cc:824] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71: Log is configured to *not* fsync() on all Append() calls
I20260924 23:10:51.226959 14902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71: No bootstrap required, opened a new log
Sep 24 23:10:51 dist-test-slave-m0z6 krb5kdc[14811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:51.229064 14902 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40524990b4494484920a2b5f044b3a71" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39035 } }
I20260924 23:10:51.229210 14902 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:51.229243 14902 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40524990b4494484920a2b5f044b3a71, State: Initialized, Role: FOLLOWER
I20260924 23:10:51.229382 14902 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40524990b4494484920a2b5f044b3a71" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39035 } }
I20260924 23:10:51.229456 14902 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:10:51.229497 14902 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:10:51.229544 14902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:51.230201 14902 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40524990b4494484920a2b5f044b3a71" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39035 } }
I20260924 23:10:51.230319 14902 leader_election.cc:302] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40524990b4494484920a2b5f044b3a71; no voters:
I20260924 23:10:51.230516 14902 leader_election.cc:288] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:10:51.230571 14907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:10:51.230729 14907 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71 [term 1 LEADER]: Becoming Leader. State: Replica: 40524990b4494484920a2b5f044b3a71, State: Running, Role: LEADER
I20260924 23:10:51.230806 14902 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:10:51.230829 14907 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40524990b4494484920a2b5f044b3a71" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39035 } }
I20260924 23:10:51.231172 14909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40524990b4494484920a2b5f044b3a71. Latest consensus state: current_term: 1 leader_uuid: "40524990b4494484920a2b5f044b3a71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40524990b4494484920a2b5f044b3a71" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39035 } } }
I20260924 23:10:51.231180 14908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40524990b4494484920a2b5f044b3a71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40524990b4494484920a2b5f044b3a71" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39035 } } }
I20260924 23:10:51.231379 14909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71 [sys.catalog]: This master's current role is: LEADER
I20260924 23:10:51.231393 14908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71 [sys.catalog]: This master's current role is: LEADER
W20260924 23:10:51.232686 14922 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:10:51.232740 14922 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:10:51.232815 14916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:10:51.232970 14916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:10:51.233206 14905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.223192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55698 (local address 127.12.212.62:39035)
0924 23:10:51.223362 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:51.223370 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:51.223395 (+ 25us) server_negotiation.cc:407] Connection header received
0924 23:10:51.223775 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:51.223781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:51.223835 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:51.223977 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:51.224604 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.225514 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.226267 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.226731 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.226831 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.229240 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:51.229244 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:51.229244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:51.229258 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:51.231495 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:51.231959 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:51.231962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:51.231963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:51.232070 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:51.232306 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:51.232307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:51.232307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:51.232555 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:51.232707 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:51.232853 (+ 146us) server_negotiation.cc:299] Negotiation successful
0924 23:10:51.232992 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":75,"threads_started":1}
I20260924 23:10:51.234483 14916 catalog_manager.cc:1347] Generated new cluster ID: 3713845d1dfc47019e2927322e2b0dd6
I20260924 23:10:51.234539 14916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:10:51.252686 14916 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:10:51.253461 14916 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:10:51.265836 14916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71: Generated new TSK 0
I20260924 23:10:51.266078 14916 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
I20260924 23:10:51.294413 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:39035
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:51.377993 14928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:51.378229 14928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:51.378259 14928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:51.379552 14928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:51.379603 14928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:51.379626 14928 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 23:10:51.379635 14928 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 23:10:51.379652 14928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:10:51.381212 14928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.212.62:39035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:51.381477 14928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:51.381719 14928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:51.387007 14933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:51.387007 14934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:51.387001 14936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:51.387164 14928 server_base.cc:1034] running on GCE node
I20260924 23:10:51.387425 14928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:51.387655 14928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:51.388813 14928 hybrid_clock.cc:648] HybridClock initialized: now 1790291451388795 us; error 36 us; skew 500 ppm
Sep 24 23:10:51 dist-test-slave-m0z6 krb5kdc[14811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291451, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:51.391381 14928 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:10:51.391826 14928 webserver.cc:460] Webserver started at http://127.12.212.1:36003/ using document root <none> and password file <none>
I20260924 23:10:51.392009 14928 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:51.392048 14928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:51.392140 14928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:51.393126 14928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "4e2623909988482fb519c0e8400fb98a"
format_stamp: "Formatted at 2026-09-24 23:10:51 on dist-test-slave-m0z6"
server_key: "df9c869cfbaeb76a6332e00e3b685ce9"
server_key_iv: "d10b6a57a9811a929aeb401aca718279"
server_key_version: "encryptionkey@0"
I20260924 23:10:51.393464 14928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "4e2623909988482fb519c0e8400fb98a"
format_stamp: "Formatted at 2026-09-24 23:10:51 on dist-test-slave-m0z6"
server_key: "df9c869cfbaeb76a6332e00e3b685ce9"
server_key_iv: "d10b6a57a9811a929aeb401aca718279"
server_key_version: "encryptionkey@0"
I20260924 23:10:51.394738 14928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:10:51.395502 14943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:51.395704 14928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:10:51.395797 14928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "4e2623909988482fb519c0e8400fb98a"
format_stamp: "Formatted at 2026-09-24 23:10:51 on dist-test-slave-m0z6"
server_key: "df9c869cfbaeb76a6332e00e3b685ce9"
server_key_iv: "d10b6a57a9811a929aeb401aca718279"
server_key_version: "encryptionkey@0"
I20260924 23:10:51.395864 14928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:51.411386 14928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:51.413478 14928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:51.413619 14928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:51.414155 14928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:10:51.414201 14928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:51.414232 14928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:10:51.414248 14928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 23:10:51 dist-test-slave-m0z6 krb5kdc[14811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291451, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:51.421140 14928 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:33967
I20260924 23:10:51.421217 15058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:33967 every 8 connection(s)
I20260924 23:10:51.421515 14928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:10:51.424228 14905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.414619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:40791 (local address 127.12.212.62:39035)
0924 23:10:51.414687 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:51.414712 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:51.414980 (+ 268us) server_negotiation.cc:407] Connection header received
0924 23:10:51.415472 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:51.415473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:51.415486 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:51.415568 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:51.416277 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.417047 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.418296 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.418513 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.418571 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.421541 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:51.421543 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:51.421543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:51.421552 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:51.422906 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:51.423296 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:51.423298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:51.423298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:51.423357 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:51.423573 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:51.423574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:51.423574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:51.423712 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:51.423769 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:51.424032 (+ 263us) server_negotiation.cc:299] Negotiation successful
0924 23:10:51.424123 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 23:10:51.424377 14952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.414601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:39035 (local address 127.12.212.1:40791)
0924 23:10:51.414842 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:51.414889 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:51.415379 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:51.415659 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:51.415664 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:51.415964 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:51.416173 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.416179 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.417139 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.417140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.418151 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.418155 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.418601 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.418601 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.418758 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.419338 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:51.419355 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:51.421332 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:51.422988 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:51.422995 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:51.422997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:51.423224 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:51.423409 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:51.423410 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:51.423410 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:51.423518 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:51.423767 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:51.423775 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:51.423916 (+ 141us) client_negotiation.cc:769] Sending connection context
0924 23:10:51.424005 (+ 89us) client_negotiation.cc:240] Negotiation successful
0924 23:10:51.424164 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":102,"threads_started":1}
I20260924 23:10:51.424875 15059 heartbeater.cc:347] Connected to a master server at 127.12.212.62:39035
I20260924 23:10:51.424955 15059 heartbeater.cc:464] Registering TS with master...
I20260924 23:10:51.425119 15059 heartbeater.cc:511] Master 127.12.212.62:39035 requested a full tablet report, sending...
I20260924 23:10:51.425688 14855 ts_manager.cc:194] Registered new tserver with Master: 4e2623909988482fb519c0e8400fb98a (127.12.212.1:33967)
I20260924 23:10:51.426553 14855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:40791
I20260924 23:10:51.429968 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 14928
I20260924 23:10:51.430058 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:10:51.430217 13136 external_mini_cluster.cc:1428] Setting key f5b6acb6d1849d404918ca24114276c3
I20260924 23:10:51.431352 15065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.425844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:39035 (local address 127.12.212.1:58083)
0924 23:10:51.425977 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:51.425988 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:51.426047 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:51.426161 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:51.426161 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:51.426346 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:51.426404 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.426405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.427197 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.427198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.427793 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.427796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.428091 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.428091 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.428169 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.428570 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:51.428576 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:51.428862 (+ 286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:51.430395 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:51.430396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:51.430397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:51.430560 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:51.430763 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:51.430764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:51.430764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:51.430824 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:51.431031 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:51.431032 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:51.431079 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 23:10:51.431137 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 23:10:51.431253 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260924 23:10:51.431370 14905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.425882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:58083 (local address 127.12.212.62:39035)
0924 23:10:51.425920 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:51.425921 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:51.426033 (+ 112us) server_negotiation.cc:407] Connection header received
0924 23:10:51.426098 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:51.426099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:51.426106 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:51.426145 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:51.426456 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.427073 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.427859 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.428037 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.428082 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.428948 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:51.428949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:51.428949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:51.428956 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:51.430324 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:51.430618 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:51.430619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:51.430619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:51.430679 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:51.430876 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:51.430877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:51.430877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:51.430980 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:51.431032 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:51.431165 (+ 133us) server_negotiation.cc:299] Negotiation successful
0924 23:10:51.431252 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for kudu/127.12.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
I20260924 23:10:51.454615 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:39035
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:51.536372 15068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:51.536604 15068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:51.536633 15068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:51.537853 15068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:51.537899 15068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:51.537920 15068 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 23:10:51.537931 15068 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 23:10:51.537949 15068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:10:51.539564 15068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.212.62:39035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:51.539855 15068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:51.540062 15068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:51.545147 15073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:51.545188 15068 server_base.cc:1034] running on GCE node
W20260924 23:10:51.545202 15076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:51.545147 15074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:51.545543 15068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:51.545804 15068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:51.547012 15068 hybrid_clock.cc:648] HybridClock initialized: now 1790291451546990 us; error 37 us; skew 500 ppm
Sep 24 23:10:51 dist-test-slave-m0z6 krb5kdc[14811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291451, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:51.549757 15068 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:10:51.550302 15068 webserver.cc:460] Webserver started at http://127.12.212.2:44151/ using document root <none> and password file <none>
I20260924 23:10:51.550552 15068 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:51.550618 15068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:51.550800 15068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:51.552254 15068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "89f6cadda86249febfd51c9858317d50"
format_stamp: "Formatted at 2026-09-24 23:10:51 on dist-test-slave-m0z6"
server_key: "daa632202769c283b17fcc49ace56f84"
server_key_iv: "b54fa316c7f744a1f5f1de331de75523"
server_key_version: "encryptionkey@0"
I20260924 23:10:51.552695 15068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "89f6cadda86249febfd51c9858317d50"
format_stamp: "Formatted at 2026-09-24 23:10:51 on dist-test-slave-m0z6"
server_key: "daa632202769c283b17fcc49ace56f84"
server_key_iv: "b54fa316c7f744a1f5f1de331de75523"
server_key_version: "encryptionkey@0"
I20260924 23:10:51.554344 15068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:10:51.555233 15083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:51.555442 15068 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 23:10:51.555537 15068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "89f6cadda86249febfd51c9858317d50"
format_stamp: "Formatted at 2026-09-24 23:10:51 on dist-test-slave-m0z6"
server_key: "daa632202769c283b17fcc49ace56f84"
server_key_iv: "b54fa316c7f744a1f5f1de331de75523"
server_key_version: "encryptionkey@0"
I20260924 23:10:51.555610 15068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:51.572727 15068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:51.575302 15068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:51.575474 15068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:51.576066 15068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:10:51.576113 15068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:51.576143 15068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:10:51.576167 15068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 23:10:51 dist-test-slave-m0z6 krb5kdc[14811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291451, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:51.583273 15068 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:46637
I20260924 23:10:51.583647 15068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:10:51.583972 15198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:46637 every 8 connection(s)
I20260924 23:10:51.586437 14905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.576558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:52133 (local address 127.12.212.62:39035)
0924 23:10:51.576625 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:51.576627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:51.576846 (+ 219us) server_negotiation.cc:407] Connection header received
0924 23:10:51.577361 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:51.577362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:51.577376 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:51.577421 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:51.578200 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.578819 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.580437 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.580655 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.580716 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.583469 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:51.583471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:51.583471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:51.583480 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:51.584888 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:51.585366 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:51.585367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:51.585367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:51.585432 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:51.585752 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:51.585753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:51.585753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:51.585890 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:51.585955 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:51.586293 (+ 338us) server_negotiation.cc:299] Negotiation successful
0924 23:10:51.586369 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 23:10:51.586673 15092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.576553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:39035 (local address 127.12.212.2:52133)
0924 23:10:51.576716 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:51.576755 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:51.577261 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:51.577505 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:51.577509 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:51.577875 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:51.578094 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.578100 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.578923 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.578924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.580334 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.580339 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.580735 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.580736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.580905 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.581511 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:51.581522 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:51.583365 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:51.584987 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:51.584994 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:51.584996 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:51.585236 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:51.585500 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:51.585501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:51.585501 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:51.585612 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:51.585985 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:51.585997 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:51.586165 (+ 168us) client_negotiation.cc:769] Sending connection context
0924 23:10:51.586262 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 23:10:51.586407 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260924 23:10:51.587179 15199 heartbeater.cc:347] Connected to a master server at 127.12.212.62:39035
I20260924 23:10:51.587270 15199 heartbeater.cc:464] Registering TS with master...
I20260924 23:10:51.587484 15199 heartbeater.cc:511] Master 127.12.212.62:39035 requested a full tablet report, sending...
I20260924 23:10:51.587901 14856 ts_manager.cc:194] Registered new tserver with Master: 89f6cadda86249febfd51c9858317d50 (127.12.212.2:46637)
I20260924 23:10:51.588405 14856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:52133
I20260924 23:10:51.589658 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 15068
I20260924 23:10:51.589758 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:10:51.589998 13136 external_mini_cluster.cc:1428] Setting key f08c180a0d43e8a99b55e66386cf45ae
I20260924 23:10:51.594832 15205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.588272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:39035 (local address 127.12.212.2:47257)
0924 23:10:51.588402 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:51.588411 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:51.588456 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:51.588697 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:51.588698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:51.588907 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:51.588989 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.588991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.589827 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.589829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.590533 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.590537 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.591014 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.591015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.591091 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.591553 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:51.591558 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:51.591991 (+ 433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:51.593708 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:51.593709 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:51.593709 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:51.593908 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:51.594118 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:51.594119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:51.594119 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:51.594173 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:51.594510 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:51.594511 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:51.594558 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 23:10:51.594613 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 23:10:51.594741 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":3,"thread_start_us":83,"threads_started":1}
I20260924 23:10:51.594871 14905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.588538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:47257 (local address 127.12.212.62:39035)
0924 23:10:51.588578 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:51.588580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:51.588587 (+ 7us) server_negotiation.cc:407] Connection header received
0924 23:10:51.588618 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:51.588619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:51.588629 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:51.588679 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:51.589075 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.589706 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.590611 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.590946 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.591011 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.592100 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:51.592102 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:51.592102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:51.592111 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:51.593563 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:51.593986 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:51.593987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:51.593988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:51.594053 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:51.594227 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:51.594228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:51.594228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:51.594443 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:51.594515 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:51.594660 (+ 145us) server_negotiation.cc:299] Negotiation successful
0924 23:10:51.594779 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.12.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
I20260924 23:10:51.618588 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:39035
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:51.702240 15208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:51.702476 15208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:51.702505 15208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:51.703871 15208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:51.703923 15208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:51.703944 15208 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 23:10:51.703955 15208 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 23:10:51.703975 15208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:10:51.705570 15208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.212.62:39035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:51.705837 15208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:51.706061 15208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:51.711257 15214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:51.711272 15213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:51.711237 15216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:51.711561 15208 server_base.cc:1034] running on GCE node
I20260924 23:10:51.711927 15208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:51.712245 15208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:51.713395 15208 hybrid_clock.cc:648] HybridClock initialized: now 1790291451713383 us; error 30 us; skew 500 ppm
Sep 24 23:10:51 dist-test-slave-m0z6 krb5kdc[14811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291451, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:51.716311 15208 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:10:51.716827 15208 webserver.cc:460] Webserver started at http://127.12.212.3:38635/ using document root <none> and password file <none>
I20260924 23:10:51.717049 15208 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:51.717095 15208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:51.717214 15208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:51.718480 15208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c"
format_stamp: "Formatted at 2026-09-24 23:10:51 on dist-test-slave-m0z6"
server_key: "807843ac706c1703765c830a9b438ca3"
server_key_iv: "79e3a691b938576f9ad2b2e1c135fb52"
server_key_version: "encryptionkey@0"
I20260924 23:10:51.718878 15208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c"
format_stamp: "Formatted at 2026-09-24 23:10:51 on dist-test-slave-m0z6"
server_key: "807843ac706c1703765c830a9b438ca3"
server_key_iv: "79e3a691b938576f9ad2b2e1c135fb52"
server_key_version: "encryptionkey@0"
I20260924 23:10:51.720225 15208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:10:51.721004 15223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:51.721246 15208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:10:51.721331 15208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c"
format_stamp: "Formatted at 2026-09-24 23:10:51 on dist-test-slave-m0z6"
server_key: "807843ac706c1703765c830a9b438ca3"
server_key_iv: "79e3a691b938576f9ad2b2e1c135fb52"
server_key_version: "encryptionkey@0"
I20260924 23:10:51.721406 15208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:51.739636 15208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:51.742177 15208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:51.742374 15208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:51.743012 15208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:10:51.743059 15208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:51.743083 15208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:10:51.743093 15208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 23:10:51 dist-test-slave-m0z6 krb5kdc[14811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291451, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:51.749737 15208 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:38479
I20260924 23:10:51.749833 15338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:38479 every 8 connection(s)
I20260924 23:10:51.750144 15208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:10:51.752938 14905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.743471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:59721 (local address 127.12.212.62:39035)
0924 23:10:51.743523 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:51.743525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:51.743758 (+ 233us) server_negotiation.cc:407] Connection header received
0924 23:10:51.744276 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:51.744279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:51.744292 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:51.744369 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:51.745259 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.746051 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.747403 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.747698 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.747775 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.750076 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:51.750078 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:51.750078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:51.750087 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:51.751539 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:51.752021 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:51.752022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:51.752022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:51.752079 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:51.752297 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:51.752297 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:51.752298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:51.752409 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:51.752467 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:51.752793 (+ 326us) server_negotiation.cc:299] Negotiation successful
0924 23:10:51.752870 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 23:10:51.753069 15232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.743449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:39035 (local address 127.12.212.3:59721)
0924 23:10:51.743622 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:51.743663 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:51.744167 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:51.744440 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:51.744445 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:51.744810 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:51.745111 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.745118 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.746153 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.746153 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.747315 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.747318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.747792 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.747793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.747931 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.748480 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:51.748491 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:51.749974 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:51.751691 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:51.751697 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:51.751699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:51.751941 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:51.752141 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:51.752142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:51.752142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:51.752251 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:51.752488 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:51.752495 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:51.752632 (+ 137us) client_negotiation.cc:769] Sending connection context
0924 23:10:51.752731 (+ 99us) client_negotiation.cc:240] Negotiation successful
0924 23:10:51.752880 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":92,"threads_started":1}
I20260924 23:10:51.753513 15339 heartbeater.cc:347] Connected to a master server at 127.12.212.62:39035
I20260924 23:10:51.753598 15339 heartbeater.cc:464] Registering TS with master...
I20260924 23:10:51.753708 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 15208
I20260924 23:10:51.753782 15339 heartbeater.cc:511] Master 127.12.212.62:39035 requested a full tablet report, sending...
I20260924 23:10:51.753806 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:10:51.753933 13136 external_mini_cluster.cc:1428] Setting key aa5269865a463d295c76a920b169a689
I20260924 23:10:51.754168 14855 ts_manager.cc:194] Registered new tserver with Master: 4ed7ae7b411c4a018b9e50d6c2a0104c (127.12.212.3:38479)
I20260924 23:10:51.754612 14855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:59721
I20260924 23:10:51.754807 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 23:10:51.760702 14905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.754620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:38711 (local address 127.12.212.62:39035)
0924 23:10:51.754796 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:51.754798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:51.755038 (+ 240us) server_negotiation.cc:407] Connection header received
0924 23:10:51.755075 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:51.755076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:51.755087 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:51.755130 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:51.755428 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.755951 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.756840 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.757030 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.757096 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.758075 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:51.758077 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:51.758077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:51.758085 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:51.759639 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:51.759971 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:51.759972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:51.759972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:51.760030 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:51.760195 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:51.760196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:51.760196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:51.760311 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:51.760368 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:51.760517 (+ 149us) server_negotiation.cc:299] Negotiation successful
0924 23:10:51.760600 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260924 23:10:51.760723 15345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.754620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:39035 (local address 127.12.212.3:38711)
0924 23:10:51.754964 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:51.754974 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:51.755028 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:51.755132 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:51.755133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:51.755304 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:51.755363 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.755364 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.756037 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.756038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.756731 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.756735 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.757108 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.757109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.757181 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.757646 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:51.757651 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:51.757983 (+ 332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:51.759728 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:51.759729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:51.759730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:51.759905 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:51.760081 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:51.760082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:51.760082 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:51.760142 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:51.760365 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:51.760366 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:51.760430 (+ 64us) client_negotiation.cc:769] Sending connection context
0924 23:10:51.760489 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 23:10:51.760614 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"spinlock_wait_cycles":219008,"thread_start_us":63,"threads_started":1}
Sep 24 23:10:51 dist-test-slave-m0z6 krb5kdc[14811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 23:10:51 dist-test-slave-m0z6 krb5kdc[14811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:51.778980 14905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.771030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55712 (local address 127.12.212.62:39035)
0924 23:10:51.771137 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:51.771140 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:51.771150 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:10:51.771176 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:51.771177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:51.771188 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:51.771287 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:51.771925 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.772457 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.773261 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.773473 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.773537 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.776221 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:51.776223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:51.776223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:51.776232 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:51.777575 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:51.777989 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:51.777990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:51.777991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:51.778053 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:51.778319 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:51.778320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:51.778321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:51.778464 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:51.778559 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:51.778833 (+ 274us) server_negotiation.cc:299] Negotiation successful
0924 23:10:51.778906 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260924 23:10:51.780930 14855 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:10:51.782467 14855 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:10:51.788888 15360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.784955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:46637 (local address 127.0.0.1:45282)
0924 23:10:51.785340 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:51.785349 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:51.785413 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:51.785626 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:51.785627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:51.785628 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:10:51.785665 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.785666 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.786614 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.786615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.787667 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.787670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.788562 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.788563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.788655 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.788669 (+ 14us) client_negotiation.cc:769] Sending connection context
0924 23:10:51.788744 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 23:10:51.788800 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":2,"thread_start_us":49,"threads_started":1}
I20260924 23:10:51.788929 15359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.785142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45282 (local address 127.12.212.2:46637)
0924 23:10:51.785286 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:51.785291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:51.785415 (+ 124us) server_negotiation.cc:407] Connection header received
0924 23:10:51.785484 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:51.785486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:51.785500 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:51.785549 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:10:51.785728 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.786540 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.787767 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.788397 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.788436 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.788495 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:51.788780 (+ 285us) server_negotiation.cc:299] Negotiation successful
0924 23:10:51.788829 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260924 23:10:51.789438 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.785159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:38479 (local address 127.0.0.1:53366)
0924 23:10:51.785509 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:51.785518 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:51.785570 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:51.785785 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:51.785786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:51.785788 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:10:51.785852 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.785854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.787183 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.787184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.788407 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.788408 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.789203 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.789204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.789273 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.789277 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:10:51.789323 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 23:10:51.789366 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":35,"thread_start_us":69,"threads_started":1}
I20260924 23:10:51.789603 15356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.784955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:33967 (local address 127.0.0.1:34132)
0924 23:10:51.785118 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:51.785204 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:51.785268 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:51.785540 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:51.785543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:51.785545 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:10:51.785619 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.785621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.787151 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.787152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.788397 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.788400 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.789381 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.789382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.789442 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.789446 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:10:51.789494 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 23:10:51.789534 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":32,"spinlock_wait_cycles":896,"thread_start_us":101,"threads_started":1}
I20260924 23:10:51.789636 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.785355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53366 (local address 127.12.212.3:38479)
0924 23:10:51.785495 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:51.785500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:51.785558 (+ 58us) server_negotiation.cc:407] Connection header received
0924 23:10:51.785636 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:51.785638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:51.785653 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:51.785719 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:10:51.785931 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.787100 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.788500 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.789116 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.789182 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.789243 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:51.789448 (+ 205us) server_negotiation.cc:299] Negotiation successful
0924 23:10:51.789546 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260924 23:10:51.789945 15358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.785063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34132 (local address 127.12.212.1:33967)
0924 23:10:51.785312 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:51.785315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:51.785326 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:10:51.785369 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:51.785371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:51.785386 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:51.785461 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:10:51.785849 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.787042 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.788483 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.789221 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.789284 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.789340 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:51.789775 (+ 435us) server_negotiation.cc:299] Negotiation successful
0924 23:10:51.789822 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":91,"threads_started":1}
I20260924 23:10:51.790122 15133 tablet_service.cc:1467] Processing CreateTablet for tablet 23b4cb02321a4022985cd47799136b88 (DEFAULT_TABLE table=test-table [id=bc895489bd184fac8b896950bd42cf6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:51.790512 15133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23b4cb02321a4022985cd47799136b88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:51.790541 14993 tablet_service.cc:1467] Processing CreateTablet for tablet 23b4cb02321a4022985cd47799136b88 (DEFAULT_TABLE table=test-table [id=bc895489bd184fac8b896950bd42cf6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:51.790884 14993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23b4cb02321a4022985cd47799136b88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:51.793622 15362 tablet_bootstrap.cc:492] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50: Bootstrap starting.
I20260924 23:10:51.794023 15363 tablet_bootstrap.cc:492] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a: Bootstrap starting.
I20260924 23:10:51.794086 15273 tablet_service.cc:1467] Processing CreateTablet for tablet 23b4cb02321a4022985cd47799136b88 (DEFAULT_TABLE table=test-table [id=bc895489bd184fac8b896950bd42cf6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:51.794294 15362 tablet_bootstrap.cc:654] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:51.794449 15273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23b4cb02321a4022985cd47799136b88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:51.794602 15363 tablet_bootstrap.cc:654] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:51.794613 15362 log.cc:824] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50: Log is configured to *not* fsync() on all Append() calls
I20260924 23:10:51.794953 15363 log.cc:824] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a: Log is configured to *not* fsync() on all Append() calls
I20260924 23:10:51.795374 15362 tablet_bootstrap.cc:492] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50: No bootstrap required, opened a new log
I20260924 23:10:51.795444 15362 ts_tablet_manager.cc:1397] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:10:51.795745 15363 tablet_bootstrap.cc:492] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a: No bootstrap required, opened a new log
I20260924 23:10:51.795812 15363 ts_tablet_manager.cc:1397] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 23:10:51.797451 15366 tablet_bootstrap.cc:492] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Bootstrap starting.
I20260924 23:10:51.797466 15362 raft_consensus.cc:348] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:51.797662 15362 raft_consensus.cc:374] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:51.797704 15362 raft_consensus.cc:729] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f6cadda86249febfd51c9858317d50, State: Initialized, Role: FOLLOWER
I20260924 23:10:51.797945 15363 raft_consensus.cc:348] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:51.798013 15362 consensus_queue.cc:260] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:51.798113 15363 raft_consensus.cc:374] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:51.798146 15363 raft_consensus.cc:729] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e2623909988482fb519c0e8400fb98a, State: Initialized, Role: FOLLOWER
I20260924 23:10:51.798326 15362 ts_tablet_manager.cc:1428] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:10:51.798321 15363 consensus_queue.cc:260] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:51.798388 15199 heartbeater.cc:503] Master 127.12.212.62:39035 was elected leader, sending a full tablet report...
I20260924 23:10:51.798578 15363 ts_tablet_manager.cc:1428] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 23:10:51.798610 15059 heartbeater.cc:503] Master 127.12.212.62:39035 was elected leader, sending a full tablet report...
I20260924 23:10:51.798729 15366 tablet_bootstrap.cc:654] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:51.799093 15366 log.cc:824] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Log is configured to *not* fsync() on all Append() calls
I20260924 23:10:51.799942 15366 tablet_bootstrap.cc:492] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c: No bootstrap required, opened a new log
I20260924 23:10:51.799999 15366 ts_tablet_manager.cc:1397] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260924 23:10:51.801543 15366 raft_consensus.cc:348] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:51.801679 15366 raft_consensus.cc:374] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:51.801700 15366 raft_consensus.cc:729] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed7ae7b411c4a018b9e50d6c2a0104c, State: Initialized, Role: FOLLOWER
I20260924 23:10:51.801781 15366 consensus_queue.cc:260] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:51.801985 15366 ts_tablet_manager.cc:1428] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 23:10:51.802062 15339 heartbeater.cc:503] Master 127.12.212.62:39035 was elected leader, sending a full tablet report...
W20260924 23:10:51.834609 15200 tablet.cc:2369] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 23:10:51.922291 15060 tablet.cc:2369] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:10:51.952591 15367 raft_consensus.cc:482] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:10:51.952761 15367 raft_consensus.cc:504] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:51.953125 15367 leader_election.cc:288] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e2623909988482fb519c0e8400fb98a (127.12.212.1:33967), 4ed7ae7b411c4a018b9e50d6c2a0104c (127.12.212.3:38479)
I20260924 23:10:51.956074 15358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.953399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:54383 (local address 127.12.212.1:33967)
0924 23:10:51.953450 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:51.953451 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:51.953616 (+ 165us) server_negotiation.cc:407] Connection header received
0924 23:10:51.953686 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:51.953687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:51.953696 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:51.953745 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:10:51.953892 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.954413 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.955421 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.955775 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.955837 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.955847 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:51.955967 (+ 120us) server_negotiation.cc:299] Negotiation successful
0924 23:10:51.956002 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 23:10:51.956127 15371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.953338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:33967 (local address 127.12.212.2:54383)
0924 23:10:51.953547 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:51.953556 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:51.953600 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:51.953756 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:51.953757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:51.953758 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:10:51.953831 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.953832 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.954497 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.954498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.955334 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.955337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.955842 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.955843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.955908 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.955912 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:10:51.955957 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 23:10:51.955996 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":58,"thread_start_us":102,"threads_started":1}
I20260924 23:10:51.956140 15092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.953338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:38479 (local address 127.12.212.2:33801)
0924 23:10:51.953396 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:51.953409 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:51.953458 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:51.953612 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:51.953613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:51.953616 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:10:51.953688 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.953690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.954343 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.954344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.955328 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.955332 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.955866 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.955866 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.955962 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.955965 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:10:51.956009 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 23:10:51.956052 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":157}
I20260924 23:10:51.956130 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.953403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:33801 (local address 127.12.212.3:38479)
0924 23:10:51.953478 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:51.953479 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:51.953513 (+ 34us) server_negotiation.cc:407] Connection header received
0924 23:10:51.953537 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:51.953538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:51.953546 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:51.953592 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:10:51.953766 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.954261 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.955426 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.955797 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.955861 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.955869 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:51.956026 (+ 157us) server_negotiation.cc:299] Negotiation successful
0924 23:10:51.956063 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 23:10:51.956486 15293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23b4cb02321a4022985cd47799136b88" candidate_uuid: "89f6cadda86249febfd51c9858317d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" is_pre_election: true
I20260924 23:10:51.956544 15013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23b4cb02321a4022985cd47799136b88" candidate_uuid: "89f6cadda86249febfd51c9858317d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e2623909988482fb519c0e8400fb98a" is_pre_election: true
I20260924 23:10:51.956656 15013 raft_consensus.cc:2454] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89f6cadda86249febfd51c9858317d50 in term 0.
I20260924 23:10:51.956657 15293 raft_consensus.cc:2454] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89f6cadda86249febfd51c9858317d50 in term 0.
I20260924 23:10:51.956892 15084 leader_election.cc:302] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed7ae7b411c4a018b9e50d6c2a0104c, 89f6cadda86249febfd51c9858317d50; no voters:
I20260924 23:10:51.957031 15367 raft_consensus.cc:2781] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:10:51.957101 15367 raft_consensus.cc:482] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:10:51.957125 15367 raft_consensus.cc:3037] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:51.958097 15367 raft_consensus.cc:504] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:51.958269 15367 leader_election.cc:288] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [CANDIDATE]: Term 1 election: Requested vote from peers 4e2623909988482fb519c0e8400fb98a (127.12.212.1:33967), 4ed7ae7b411c4a018b9e50d6c2a0104c (127.12.212.3:38479)
I20260924 23:10:51.958448 15013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23b4cb02321a4022985cd47799136b88" candidate_uuid: "89f6cadda86249febfd51c9858317d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e2623909988482fb519c0e8400fb98a"
I20260924 23:10:51.958527 15013 raft_consensus.cc:3037] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:51.958770 15293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23b4cb02321a4022985cd47799136b88" candidate_uuid: "89f6cadda86249febfd51c9858317d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c"
I20260924 23:10:51.958861 15293 raft_consensus.cc:3037] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:51.959323 15013 raft_consensus.cc:2454] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89f6cadda86249febfd51c9858317d50 in term 1.
I20260924 23:10:51.959635 15293 raft_consensus.cc:2454] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89f6cadda86249febfd51c9858317d50 in term 1.
I20260924 23:10:51.959824 15084 leader_election.cc:302] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed7ae7b411c4a018b9e50d6c2a0104c, 89f6cadda86249febfd51c9858317d50; no voters:
I20260924 23:10:51.959939 15367 raft_consensus.cc:2781] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:10:51.960100 15367 raft_consensus.cc:686] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [term 1 LEADER]: Becoming Leader. State: Replica: 89f6cadda86249febfd51c9858317d50, State: Running, Role: LEADER
I20260924 23:10:51.960263 15367 consensus_queue.cc:237] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:51.961347 14855 catalog_manager.cc:5495] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89f6cadda86249febfd51c9858317d50 (127.12.212.2). New cstate: current_term: 1 leader_uuid: "89f6cadda86249febfd51c9858317d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } health_report { overall_health: UNKNOWN } } }
Sep 24 23:10:51 dist-test-slave-m0z6 krb5kdc[14811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291451, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:51.994560 15379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.987483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:39035 (local address 127.0.0.1:55716)
0924 23:10:51.987650 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:51.987662 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:51.987741 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:51.987890 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:51.987891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:51.988219 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:51.988269 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.988272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.988920 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.988921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.989489 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:51.989491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.989835 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.989836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:51.989885 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.990427 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:51.990432 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:51.991955 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:51.993422 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:51.993430 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:51.993432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:51.993633 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:51.993871 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:51.993872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:51.993872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:51.993940 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:51.994201 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:51.994213 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:51.994267 (+ 54us) client_negotiation.cc:769] Sending connection context
0924 23:10:51.994335 (+ 68us) client_negotiation.cc:240] Negotiation successful
0924 23:10:51.994454 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":2,"thread_start_us":66,"threads_started":1}
I20260924 23:10:51.994606 14905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:51.987561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55716 (local address 127.12.212.62:39035)
0924 23:10:51.987647 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:51.987649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:51.987732 (+ 83us) server_negotiation.cc:407] Connection header received
0924 23:10:51.987802 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:51.987803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:51.987811 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:51.987860 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:51.988349 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.988831 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.989565 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:51.989755 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:51.989807 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:51.992041 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:51.992042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:51.992043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:51.992049 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:51.993336 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:51.993719 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:51.993720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:51.993720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:51.993780 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:51.994001 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:51.994001 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:51.994002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:51.994114 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:51.994171 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:51.994358 (+ 187us) server_negotiation.cc:299] Negotiation successful
0924 23:10:51.994504 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":73,"server-negotiator.queue_time_us":60}
I20260924 23:10:51.996770 14855 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:55716:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 23:10:51.996963 14855 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:10:51.998433 14993 tablet_service.cc:1467] Processing CreateTablet for tablet f418eda2a0e24e4e8a8916c64301f421 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f78e6492e1274c90a84c5debf514941d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 23:10:51.998433 15133 tablet_service.cc:1467] Processing CreateTablet for tablet f418eda2a0e24e4e8a8916c64301f421 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f78e6492e1274c90a84c5debf514941d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 23:10:51.998514 15273 tablet_service.cc:1467] Processing CreateTablet for tablet f418eda2a0e24e4e8a8916c64301f421 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f78e6492e1274c90a84c5debf514941d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 23:10:51.998610 15133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f418eda2a0e24e4e8a8916c64301f421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:51.998608 14993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f418eda2a0e24e4e8a8916c64301f421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:51.998652 15273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f418eda2a0e24e4e8a8916c64301f421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:52.000098 15363 tablet_bootstrap.cc:492] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a: Bootstrap starting.
I20260924 23:10:52.000111 15362 tablet_bootstrap.cc:492] T f418eda2a0e24e4e8a8916c64301f421 P 89f6cadda86249febfd51c9858317d50: Bootstrap starting.
I20260924 23:10:52.000111 15366 tablet_bootstrap.cc:492] T f418eda2a0e24e4e8a8916c64301f421 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Bootstrap starting.
I20260924 23:10:52.000443 15363 tablet_bootstrap.cc:654] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:52.000548 15362 tablet_bootstrap.cc:654] T f418eda2a0e24e4e8a8916c64301f421 P 89f6cadda86249febfd51c9858317d50: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:52.000612 15366 tablet_bootstrap.cc:654] T f418eda2a0e24e4e8a8916c64301f421 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Neither blocks nor log segments found. Creating new log.
W20260924 23:10:52.000685 15340 tablet.cc:2369] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:10:52.001205 15366 tablet_bootstrap.cc:492] T f418eda2a0e24e4e8a8916c64301f421 P 4ed7ae7b411c4a018b9e50d6c2a0104c: No bootstrap required, opened a new log
I20260924 23:10:52.001267 15366 ts_tablet_manager.cc:1397] T f418eda2a0e24e4e8a8916c64301f421 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 23:10:52.001364 15363 tablet_bootstrap.cc:492] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a: No bootstrap required, opened a new log
I20260924 23:10:52.001433 15363 ts_tablet_manager.cc:1397] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 23:10:52.001416 15366 raft_consensus.cc:348] T f418eda2a0e24e4e8a8916c64301f421 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:52.001500 15366 raft_consensus.cc:374] T f418eda2a0e24e4e8a8916c64301f421 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:52.001525 15366 raft_consensus.cc:729] T f418eda2a0e24e4e8a8916c64301f421 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed7ae7b411c4a018b9e50d6c2a0104c, State: Initialized, Role: FOLLOWER
I20260924 23:10:52.001576 15366 consensus_queue.cc:260] T f418eda2a0e24e4e8a8916c64301f421 P 4ed7ae7b411c4a018b9e50d6c2a0104c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:52.001578 15363 raft_consensus.cc:348] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:52.001646 15363 raft_consensus.cc:374] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:52.001657 15366 ts_tablet_manager.cc:1428] T f418eda2a0e24e4e8a8916c64301f421 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 23:10:52.001663 15363 raft_consensus.cc:729] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e2623909988482fb519c0e8400fb98a, State: Initialized, Role: FOLLOWER
I20260924 23:10:52.001701 15363 consensus_queue.cc:260] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:52.001725 15370 tablet_replica.cc:439] T f418eda2a0e24e4e8a8916c64301f421 P 4ed7ae7b411c4a018b9e50d6c2a0104c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } } }
I20260924 23:10:52.001802 15370 tablet_replica.cc:442] T f418eda2a0e24e4e8a8916c64301f421 P 4ed7ae7b411c4a018b9e50d6c2a0104c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:10:52.001837 15363 ts_tablet_manager.cc:1428] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:52.001874 15362 tablet_bootstrap.cc:492] T f418eda2a0e24e4e8a8916c64301f421 P 89f6cadda86249febfd51c9858317d50: No bootstrap required, opened a new log
I20260924 23:10:52.001921 15362 ts_tablet_manager.cc:1397] T f418eda2a0e24e4e8a8916c64301f421 P 89f6cadda86249febfd51c9858317d50: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:10:52.002046 15362 raft_consensus.cc:348] T f418eda2a0e24e4e8a8916c64301f421 P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:52.002127 15362 raft_consensus.cc:374] T f418eda2a0e24e4e8a8916c64301f421 P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:52.002156 15362 raft_consensus.cc:729] T f418eda2a0e24e4e8a8916c64301f421 P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f6cadda86249febfd51c9858317d50, State: Initialized, Role: FOLLOWER
I20260924 23:10:52.002193 15362 consensus_queue.cc:260] T f418eda2a0e24e4e8a8916c64301f421 P 89f6cadda86249febfd51c9858317d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:52.002290 15362 ts_tablet_manager.cc:1428] T f418eda2a0e24e4e8a8916c64301f421 P 89f6cadda86249febfd51c9858317d50: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:52.002375 15367 tablet_replica.cc:439] T f418eda2a0e24e4e8a8916c64301f421 P 89f6cadda86249febfd51c9858317d50: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } } }
I20260924 23:10:52.002421 15368 tablet_replica.cc:439] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } } }
I20260924 23:10:52.002465 15367 tablet_replica.cc:442] T f418eda2a0e24e4e8a8916c64301f421 P 89f6cadda86249febfd51c9858317d50: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:10:52.002490 15368 tablet_replica.cc:442] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:10:52.049284 15368 raft_consensus.cc:482] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:10:52.049386 15368 raft_consensus.cc:504] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:52.049780 15368 leader_election.cc:288] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89f6cadda86249febfd51c9858317d50 (127.12.212.2:46637), 4ed7ae7b411c4a018b9e50d6c2a0104c (127.12.212.3:38479)
I20260924 23:10:52.052886 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:52.050289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:46717 (local address 127.12.212.3:38479)
0924 23:10:52.050331 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:52.050333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:52.050374 (+ 41us) server_negotiation.cc:407] Connection header received
0924 23:10:52.050438 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:52.050439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:52.050447 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:52.050494 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:10:52.050670 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.051150 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.052200 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.052555 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.052617 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:52.052635 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:52.052768 (+ 133us) server_negotiation.cc:299] Negotiation successful
0924 23:10:52.052807 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 23:10:52.052902 15384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:52.050214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:38479 (local address 127.12.212.1:46717)
0924 23:10:52.050323 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:52.050332 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:52.050388 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:52.050502 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:52.050503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:52.050505 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:10:52.050569 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:52.050571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.051225 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.051226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:52.052100 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:52.052103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.052645 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.052646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:52.052708 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:52.052711 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:10:52.052756 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 23:10:52.052798 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":24,"thread_start_us":43,"threads_started":1}
I20260924 23:10:52.053165 15359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:52.050097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:49967 (local address 127.12.212.2:46637)
0924 23:10:52.050179 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:52.050182 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:52.050256 (+ 74us) server_negotiation.cc:407] Connection header received
0924 23:10:52.050323 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:52.050323 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:52.050333 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:52.050383 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:10:52.050523 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.051134 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.052392 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.052770 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.052833 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:52.052841 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:52.053053 (+ 212us) server_negotiation.cc:299] Negotiation successful
0924 23:10:52.053100 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 23:10:52.053175 15293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f418eda2a0e24e4e8a8916c64301f421" candidate_uuid: "4e2623909988482fb519c0e8400fb98a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" is_pre_election: true
I20260924 23:10:52.053272 15293 raft_consensus.cc:2454] T f418eda2a0e24e4e8a8916c64301f421 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e2623909988482fb519c0e8400fb98a in term 0.
I20260924 23:10:52.053375 15383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:52.050015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:46637 (local address 127.12.212.1:49967)
0924 23:10:52.050175 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:52.050188 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:52.050268 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:52.050394 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:52.050395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:52.050397 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:10:52.050470 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:52.050472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.051214 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.051215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:52.052304 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:52.052307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.052885 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.052886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:52.052962 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:52.052966 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:10:52.053255 (+ 289us) client_negotiation.cc:240] Negotiation successful
0924 23:10:52.053299 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":29,"thread_start_us":64,"threads_started":1}
I20260924 23:10:52.053465 14944 leader_election.cc:302] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e2623909988482fb519c0e8400fb98a, 4ed7ae7b411c4a018b9e50d6c2a0104c; no voters:
I20260924 23:10:52.053615 15368 raft_consensus.cc:2781] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:10:52.053665 15368 raft_consensus.cc:482] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:10:52.053686 15368 raft_consensus.cc:3037] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:52.053823 15153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f418eda2a0e24e4e8a8916c64301f421" candidate_uuid: "4e2623909988482fb519c0e8400fb98a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f6cadda86249febfd51c9858317d50" is_pre_election: true
I20260924 23:10:52.053993 15153 raft_consensus.cc:2454] T f418eda2a0e24e4e8a8916c64301f421 P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e2623909988482fb519c0e8400fb98a in term 0.
I20260924 23:10:52.054508 15368 raft_consensus.cc:504] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:52.054628 15368 leader_election.cc:288] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a [CANDIDATE]: Term 1 election: Requested vote from peers 89f6cadda86249febfd51c9858317d50 (127.12.212.2:46637), 4ed7ae7b411c4a018b9e50d6c2a0104c (127.12.212.3:38479)
I20260924 23:10:52.054908 15153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f418eda2a0e24e4e8a8916c64301f421" candidate_uuid: "4e2623909988482fb519c0e8400fb98a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f6cadda86249febfd51c9858317d50"
I20260924 23:10:52.054919 15293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f418eda2a0e24e4e8a8916c64301f421" candidate_uuid: "4e2623909988482fb519c0e8400fb98a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c"
I20260924 23:10:52.055002 15153 raft_consensus.cc:3037] T f418eda2a0e24e4e8a8916c64301f421 P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:52.055001 15293 raft_consensus.cc:3037] T f418eda2a0e24e4e8a8916c64301f421 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:52.055783 15293 raft_consensus.cc:2454] T f418eda2a0e24e4e8a8916c64301f421 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e2623909988482fb519c0e8400fb98a in term 1.
I20260924 23:10:52.055830 15153 raft_consensus.cc:2454] T f418eda2a0e24e4e8a8916c64301f421 P 89f6cadda86249febfd51c9858317d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e2623909988482fb519c0e8400fb98a in term 1.
I20260924 23:10:52.056030 14944 leader_election.cc:302] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e2623909988482fb519c0e8400fb98a, 89f6cadda86249febfd51c9858317d50; no voters:
I20260924 23:10:52.056109 15368 raft_consensus.cc:2781] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:10:52.056254 15368 raft_consensus.cc:686] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a [term 1 LEADER]: Becoming Leader. State: Replica: 4e2623909988482fb519c0e8400fb98a, State: Running, Role: LEADER
I20260924 23:10:52.056384 15368 consensus_queue.cc:237] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:52.056907 15385 tablet_replica.cc:439] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a: TxnStatusTablet state changed. Reason: New leader 4e2623909988482fb519c0e8400fb98a. Latest consensus state: current_term: 1 leader_uuid: "4e2623909988482fb519c0e8400fb98a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } } }
I20260924 23:10:52.057027 15385 tablet_replica.cc:442] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a: This TxnStatusTablet replica's current role is: LEADER
I20260924 23:10:52.057303 14856 catalog_manager.cc:5495] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e2623909988482fb519c0e8400fb98a (127.12.212.1). New cstate: current_term: 1 leader_uuid: "4e2623909988482fb519c0e8400fb98a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } health_report { overall_health: UNKNOWN } } }
I20260924 23:10:52.057323 15387 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 23:10:52.057549 15387 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 23:10:52.077464 15358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:52.074769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34138 (local address 127.12.212.1:33967)
0924 23:10:52.074815 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:52.074817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:52.074826 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:10:52.074931 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:52.074932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:52.074948 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:52.075004 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:10:52.075137 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.075618 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.076729 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.076916 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.076971 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:52.077149 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:10:52.077266 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:10:52.077323 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:52.077376 (+ 53us) server_negotiation.cc:299] Negotiation successful
0924 23:10:52.077403 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 23:10:52.077510 15379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:52.074653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:33967 (local address 127.0.0.1:34138)
0924 23:10:52.074778 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:52.074790 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:52.074843 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:52.075024 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:52.075025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:52.075026 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 23:10:52.075082 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:52.075084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.075681 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.075682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:52.076604 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:52.076607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.076979 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.076980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:52.077044 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:52.077062 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 23:10:52.077321 (+ 259us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 23:10:52.077326 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 23:10:52.077366 (+ 40us) client_negotiation.cc:240] Negotiation successful
0924 23:10:52.077412 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260924 23:10:52.079649 15293 raft_consensus.cc:1261] T f418eda2a0e24e4e8a8916c64301f421 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 1 FOLLOWER]: Refusing update from remote peer 4e2623909988482fb519c0e8400fb98a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:10:52.079664 15153 raft_consensus.cc:1261] T f418eda2a0e24e4e8a8916c64301f421 P 89f6cadda86249febfd51c9858317d50 [term 1 FOLLOWER]: Refusing update from remote peer 4e2623909988482fb519c0e8400fb98a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:10:52.080001 15368 consensus_queue.cc:1035] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:10:52.080147 15385 consensus_queue.cc:1035] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a [LEADER]: Connected to new peer: Peer: permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:10:52.081344 15367 tablet_replica.cc:439] T f418eda2a0e24e4e8a8916c64301f421 P 89f6cadda86249febfd51c9858317d50: TxnStatusTablet state changed. Reason: New leader 4e2623909988482fb519c0e8400fb98a. Latest consensus state: current_term: 1 leader_uuid: "4e2623909988482fb519c0e8400fb98a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } } }
I20260924 23:10:52.081347 15370 tablet_replica.cc:439] T f418eda2a0e24e4e8a8916c64301f421 P 4ed7ae7b411c4a018b9e50d6c2a0104c: TxnStatusTablet state changed. Reason: New leader 4e2623909988482fb519c0e8400fb98a. Latest consensus state: current_term: 1 leader_uuid: "4e2623909988482fb519c0e8400fb98a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } } }
I20260924 23:10:52.081439 15370 tablet_replica.cc:442] T f418eda2a0e24e4e8a8916c64301f421 P 4ed7ae7b411c4a018b9e50d6c2a0104c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:10:52.081439 15367 tablet_replica.cc:442] T f418eda2a0e24e4e8a8916c64301f421 P 89f6cadda86249febfd51c9858317d50: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:10:52.082479 15385 tablet_replica.cc:439] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4e2623909988482fb519c0e8400fb98a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } } }
I20260924 23:10:52.082569 15385 tablet_replica.cc:442] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a: This TxnStatusTablet replica's current role is: LEADER
I20260924 23:10:52.083119 15370 tablet_replica.cc:439] T f418eda2a0e24e4e8a8916c64301f421 P 4ed7ae7b411c4a018b9e50d6c2a0104c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4e2623909988482fb519c0e8400fb98a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } } }
I20260924 23:10:52.083215 15370 tablet_replica.cc:442] T f418eda2a0e24e4e8a8916c64301f421 P 4ed7ae7b411c4a018b9e50d6c2a0104c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:10:52.083716 15368 tablet_replica.cc:439] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4e2623909988482fb519c0e8400fb98a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } } }
I20260924 23:10:52.083796 15368 tablet_replica.cc:442] T f418eda2a0e24e4e8a8916c64301f421 P 4e2623909988482fb519c0e8400fb98a: This TxnStatusTablet replica's current role is: LEADER
I20260924 23:10:52.084591 15367 tablet_replica.cc:439] T f418eda2a0e24e4e8a8916c64301f421 P 89f6cadda86249febfd51c9858317d50: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4e2623909988482fb519c0e8400fb98a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } } }
I20260924 23:10:52.084673 15367 tablet_replica.cc:442] T f418eda2a0e24e4e8a8916c64301f421 P 89f6cadda86249febfd51c9858317d50: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:10:52.087440 15359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:52.084592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45296 (local address 127.12.212.2:46637)
0924 23:10:52.084659 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:52.084660 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:52.084669 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:10:52.084698 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:52.084699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:52.084718 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:52.084773 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:10:52.084908 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.085448 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.086653 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.086906 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.086959 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:52.087131 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:10:52.087244 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:10:52.087300 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:52.087360 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 23:10:52.087380 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 23:10:52.092335 15358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:52.090149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:46181 (local address 127.12.212.1:33967)
0924 23:10:52.090198 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:52.090207 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:52.090216 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:10:52.090261 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:52.090262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:52.090272 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:52.090321 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:10:52.090459 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.090982 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.091704 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.091885 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.091940 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:52.092108 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:10:52.092137 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:10:52.092176 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:52.092236 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 23:10:52.092255 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 23:10:52.092335 15205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:52.090080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:33967 (local address 127.12.212.2:46181)
0924 23:10:52.090158 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:52.090170 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:52.090211 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:52.090333 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:52.090334 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:52.090335 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 23:10:52.090401 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:52.090403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.091052 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.091054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:52.091625 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:52.091627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.091946 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.091947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:52.092012 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:52.092025 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 23:10:52.092186 (+ 161us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 23:10:52.092188 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 23:10:52.092225 (+ 37us) client_negotiation.cc:240] Negotiation successful
0924 23:10:52.092267 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 23:10:52.094643 15293 raft_consensus.cc:1261] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 1 FOLLOWER]: Refusing update from remote peer 89f6cadda86249febfd51c9858317d50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:10:52.094762 15013 raft_consensus.cc:1261] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a [term 1 FOLLOWER]: Refusing update from remote peer 89f6cadda86249febfd51c9858317d50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:10:52.094924 15367 consensus_queue.cc:1035] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:10:52.095110 15372 consensus_queue.cc:1035] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:10:52.096450 15392 mvcc.cc:205] Tried to move back new op lower bound from 7333033787778052096 to 7333033787229982720. Current Snapshot: MvccSnapshot[applied={T|T < 7333033787778052096}]
I20260924 23:10:52.096903 15390 mvcc.cc:205] Tried to move back new op lower bound from 7333033787778052096 to 7333033787229982720. Current Snapshot: MvccSnapshot[applied={T|T < 7333033787778052096}]
I20260924 23:10:52.097870 15391 mvcc.cc:205] Tried to move back new op lower bound from 7333033787778052096 to 7333033787229982720. Current Snapshot: MvccSnapshot[applied={T|T < 7333033787778052096}]
I20260924 23:10:52.104285 15359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:52.101782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:45461 (local address 127.12.212.2:46637)
0924 23:10:52.101818 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:52.101820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:52.101975 (+ 155us) server_negotiation.cc:407] Connection header received
0924 23:10:52.102017 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:52.102018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:52.102027 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:52.102073 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:10:52.102201 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.102666 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.103636 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.103849 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.103901 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:52.104054 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:10:52.104088 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:10:52.104129 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:52.104186 (+ 57us) server_negotiation.cc:299] Negotiation successful
0924 23:10:52.104206 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 23:10:52.104308 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:52.101750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:46637 (local address 127.12.212.1:45461)
0924 23:10:52.101911 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:52.101920 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:52.101973 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:52.102074 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:52.102075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:52.102076 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 23:10:52.102145 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:52.102147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.102777 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.102778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:52.103579 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:52.103582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:52.103912 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:52.103913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:52.103977 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:52.103992 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 23:10:52.104130 (+ 138us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 23:10:52.104132 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 23:10:52.104182 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 23:10:52.104228 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260924 23:10:52.113595 14856 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:10:52.113703 14856 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:10:52.114446 14856 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71: Sending DeleteTablet for 3 replicas of tablet 23b4cb02321a4022985cd47799136b88
I20260924 23:10:52.114764 15133 tablet_service.cc:1514] Processing DeleteTablet for tablet 23b4cb02321a4022985cd47799136b88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:10:52 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:45282
I20260924 23:10:52.114997 15273 tablet_service.cc:1514] Processing DeleteTablet for tablet 23b4cb02321a4022985cd47799136b88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:10:52 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:53366
I20260924 23:10:52.115131 14856 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:10:52.115279 14856 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:10:52.115501 14993 tablet_service.cc:1514] Processing DeleteTablet for tablet 23b4cb02321a4022985cd47799136b88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:10:52 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:34132
I20260924 23:10:52.116072 15408 tablet_replica.cc:330] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50: stopping tablet replica
I20260924 23:10:52.116221 15409 tablet_replica.cc:330] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c: stopping tablet replica
I20260924 23:10:52.116377 15408 raft_consensus.cc:2229] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:10:52.116547 15409 raft_consensus.cc:2229] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:10:52.116609 15133 tablet_service.cc:1467] Processing CreateTablet for tablet 36441591f5f44db6b2ee687a0e3cec0b (DEFAULT_TABLE table=test-table [id=56ee6b52dae4477b86ed4355420b44f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:52.116724 15133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36441591f5f44db6b2ee687a0e3cec0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:52.116884 15408 raft_consensus.cc:2258] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:10:52.117000 15409 raft_consensus.cc:2258] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:10:52.117238 15410 tablet_replica.cc:330] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a: stopping tablet replica
I20260924 23:10:52.117383 15410 raft_consensus.cc:2229] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:10:52.117479 15410 raft_consensus.cc:2258] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:10:52.117996 15362 tablet_bootstrap.cc:492] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50: Bootstrap starting.
I20260924 23:10:52.118259 15273 tablet_service.cc:1467] Processing CreateTablet for tablet 36441591f5f44db6b2ee687a0e3cec0b (DEFAULT_TABLE table=test-table [id=56ee6b52dae4477b86ed4355420b44f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:52.118348 15273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36441591f5f44db6b2ee687a0e3cec0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:52.118402 15362 tablet_bootstrap.cc:654] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:52.119558 15409 ts_tablet_manager.cc:1905] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:10:52.119812 15408 ts_tablet_manager.cc:1905] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:10:52.120437 14993 tablet_service.cc:1467] Processing CreateTablet for tablet 36441591f5f44db6b2ee687a0e3cec0b (DEFAULT_TABLE table=test-table [id=56ee6b52dae4477b86ed4355420b44f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:52.120556 14993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36441591f5f44db6b2ee687a0e3cec0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:52.120191 15366 tablet_bootstrap.cc:492] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c: Bootstrap starting.
I20260924 23:10:52.121851 15363 tablet_bootstrap.cc:492] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a: Bootstrap starting.
I20260924 23:10:52.122239 15366 tablet_bootstrap.cc:654] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:52.122288 15363 tablet_bootstrap.cc:654] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:52.122982 15408 ts_tablet_manager.cc:1918] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 23:10:52.123138 15409 ts_tablet_manager.cc:1918] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 23:10:52.123247 15409 log.cc:1196] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/23b4cb02321a4022985cd47799136b88
I20260924 23:10:52.123346 15410 ts_tablet_manager.cc:1905] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:10:52.123605 15409 ts_tablet_manager.cc:1939] T 23b4cb02321a4022985cd47799136b88 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Deleting consensus metadata
I20260924 23:10:52.123729 15408 log.cc:1196] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/23b4cb02321a4022985cd47799136b88
I20260924 23:10:52.124053 15408 ts_tablet_manager.cc:1939] T 23b4cb02321a4022985cd47799136b88 P 89f6cadda86249febfd51c9858317d50: Deleting consensus metadata
I20260924 23:10:52.124501 14840 catalog_manager.cc:4841] TS 4ed7ae7b411c4a018b9e50d6c2a0104c (127.12.212.3:38479): tablet 23b4cb02321a4022985cd47799136b88 (table test-table [id=bc895489bd184fac8b896950bd42cf6e]) successfully deleted
I20260924 23:10:52.124974 14840 catalog_manager.cc:4841] TS 89f6cadda86249febfd51c9858317d50 (127.12.212.2:46637): tablet 23b4cb02321a4022985cd47799136b88 (table test-table [id=bc895489bd184fac8b896950bd42cf6e]) successfully deleted
I20260924 23:10:52.125118 15362 tablet_bootstrap.cc:492] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50: No bootstrap required, opened a new log
I20260924 23:10:52.125154 15362 ts_tablet_manager.cc:1397] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260924 23:10:52.125293 15362 raft_consensus.cc:348] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:52.125358 15362 raft_consensus.cc:374] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:52.125376 15362 raft_consensus.cc:729] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f6cadda86249febfd51c9858317d50, State: Initialized, Role: FOLLOWER
I20260924 23:10:52.125417 15362 consensus_queue.cc:260] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:52.125501 15362 ts_tablet_manager.cc:1428] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:52.125488 15410 ts_tablet_manager.cc:1918] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 23:10:52.126022 15410 log.cc:1196] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/wal/wals/23b4cb02321a4022985cd47799136b88
I20260924 23:10:52.126309 15410 ts_tablet_manager.cc:1939] T 23b4cb02321a4022985cd47799136b88 P 4e2623909988482fb519c0e8400fb98a: Deleting consensus metadata
I20260924 23:10:52.126662 14841 catalog_manager.cc:4841] TS 4e2623909988482fb519c0e8400fb98a (127.12.212.1:33967): tablet 23b4cb02321a4022985cd47799136b88 (table test-table [id=bc895489bd184fac8b896950bd42cf6e]) successfully deleted
I20260924 23:10:52.127092 15366 tablet_bootstrap.cc:492] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c: No bootstrap required, opened a new log
I20260924 23:10:52.127142 15366 ts_tablet_manager.cc:1397] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260924 23:10:52.127267 15363 tablet_bootstrap.cc:492] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a: No bootstrap required, opened a new log
I20260924 23:10:52.127321 15363 ts_tablet_manager.cc:1397] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260924 23:10:52.127290 15366 raft_consensus.cc:348] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:52.127352 15366 raft_consensus.cc:374] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:52.127377 15366 raft_consensus.cc:729] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed7ae7b411c4a018b9e50d6c2a0104c, State: Initialized, Role: FOLLOWER
I20260924 23:10:52.127416 15366 consensus_queue.cc:260] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:52.127457 15363 raft_consensus.cc:348] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:52.127501 15366 ts_tablet_manager.cc:1428] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 23:10:52.127528 15363 raft_consensus.cc:374] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:52.127552 15363 raft_consensus.cc:729] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e2623909988482fb519c0e8400fb98a, State: Initialized, Role: FOLLOWER
I20260924 23:10:52.127595 15363 consensus_queue.cc:260] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:52.127681 15363 ts_tablet_manager.cc:1428] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 23:10:52.165781 15372 raft_consensus.cc:482] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:10:52.165886 15372 raft_consensus.cc:504] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:52.166056 15372 leader_election.cc:288] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e2623909988482fb519c0e8400fb98a (127.12.212.1:33967), 4ed7ae7b411c4a018b9e50d6c2a0104c (127.12.212.3:38479)
I20260924 23:10:52.166276 15013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36441591f5f44db6b2ee687a0e3cec0b" candidate_uuid: "89f6cadda86249febfd51c9858317d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e2623909988482fb519c0e8400fb98a" is_pre_election: true
I20260924 23:10:52.166255 15293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36441591f5f44db6b2ee687a0e3cec0b" candidate_uuid: "89f6cadda86249febfd51c9858317d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" is_pre_election: true
I20260924 23:10:52.166355 15293 raft_consensus.cc:2454] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89f6cadda86249febfd51c9858317d50 in term 0.
I20260924 23:10:52.166355 15013 raft_consensus.cc:2454] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89f6cadda86249febfd51c9858317d50 in term 0.
I20260924 23:10:52.166514 15085 leader_election.cc:302] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e2623909988482fb519c0e8400fb98a, 89f6cadda86249febfd51c9858317d50; no voters:
I20260924 23:10:52.166625 15372 raft_consensus.cc:2781] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:10:52.166651 15372 raft_consensus.cc:482] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:10:52.166667 15372 raft_consensus.cc:3037] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:52.167390 15372 raft_consensus.cc:504] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:52.167518 15372 leader_election.cc:288] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [CANDIDATE]: Term 1 election: Requested vote from peers 4e2623909988482fb519c0e8400fb98a (127.12.212.1:33967), 4ed7ae7b411c4a018b9e50d6c2a0104c (127.12.212.3:38479)
I20260924 23:10:52.167716 15293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36441591f5f44db6b2ee687a0e3cec0b" candidate_uuid: "89f6cadda86249febfd51c9858317d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c"
I20260924 23:10:52.167790 15293 raft_consensus.cc:3037] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:52.167752 15013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36441591f5f44db6b2ee687a0e3cec0b" candidate_uuid: "89f6cadda86249febfd51c9858317d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e2623909988482fb519c0e8400fb98a"
I20260924 23:10:52.167819 15013 raft_consensus.cc:3037] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:52.168574 15013 raft_consensus.cc:2454] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89f6cadda86249febfd51c9858317d50 in term 1.
I20260924 23:10:52.168574 15293 raft_consensus.cc:2454] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89f6cadda86249febfd51c9858317d50 in term 1.
I20260924 23:10:52.168768 15085 leader_election.cc:302] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e2623909988482fb519c0e8400fb98a, 89f6cadda86249febfd51c9858317d50; no voters:
I20260924 23:10:52.168939 15372 raft_consensus.cc:2781] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:10:52.168980 15372 raft_consensus.cc:686] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [term 1 LEADER]: Becoming Leader. State: Replica: 89f6cadda86249febfd51c9858317d50, State: Running, Role: LEADER
I20260924 23:10:52.169020 15372 consensus_queue.cc:237] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:52.169524 14855 catalog_manager.cc:5495] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89f6cadda86249febfd51c9858317d50 (127.12.212.2). New cstate: current_term: 1 leader_uuid: "89f6cadda86249febfd51c9858317d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } health_report { overall_health: UNKNOWN } } }
I20260924 23:10:52.187873 15293 raft_consensus.cc:1261] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 1 FOLLOWER]: Refusing update from remote peer 89f6cadda86249febfd51c9858317d50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:10:52.188122 15367 consensus_queue.cc:1035] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:10:52.188524 15013 raft_consensus.cc:1261] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a [term 1 FOLLOWER]: Refusing update from remote peer 89f6cadda86249febfd51c9858317d50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:10:52.188725 15367 consensus_queue.cc:1035] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 23:10:52.195123 15392 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260924 23:10:53.730751 14855 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:10:53.730880 14855 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:10:53.731899 14855 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71: Sending DeleteTablet for 3 replicas of tablet 36441591f5f44db6b2ee687a0e3cec0b
I20260924 23:10:53.732208 14993 tablet_service.cc:1514] Processing DeleteTablet for tablet 36441591f5f44db6b2ee687a0e3cec0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:10:53 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:34132
I20260924 23:10:53.732215 15273 tablet_service.cc:1514] Processing DeleteTablet for tablet 36441591f5f44db6b2ee687a0e3cec0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:10:53 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:53366
I20260924 23:10:53.732211 15133 tablet_service.cc:1514] Processing DeleteTablet for tablet 36441591f5f44db6b2ee687a0e3cec0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:10:53 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:45282
I20260924 23:10:53.735211 15439 tablet_replica.cc:330] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a: stopping tablet replica
I20260924 23:10:53.735306 15439 raft_consensus.cc:2229] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:10:53.735361 15439 raft_consensus.cc:2258] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:10:53.735489 15438 tablet_replica.cc:330] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c: stopping tablet replica
I20260924 23:10:53.735595 15438 raft_consensus.cc:2229] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:10:53.735646 15438 raft_consensus.cc:2258] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:10:53.735657 15439 ts_tablet_manager.cc:1905] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:10:53.735922 15438 ts_tablet_manager.cc:1905] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:10:53.737174 15439 ts_tablet_manager.cc:1918] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 23:10:53.737236 15439 log.cc:1196] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/wal/wals/36441591f5f44db6b2ee687a0e3cec0b
I20260924 23:10:53.737440 15439 ts_tablet_manager.cc:1939] T 36441591f5f44db6b2ee687a0e3cec0b P 4e2623909988482fb519c0e8400fb98a: Deleting consensus metadata
I20260924 23:10:53.737587 15438 ts_tablet_manager.cc:1918] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 23:10:53.737658 15438 log.cc:1196] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/36441591f5f44db6b2ee687a0e3cec0b
I20260924 23:10:53.737871 15438 ts_tablet_manager.cc:1939] T 36441591f5f44db6b2ee687a0e3cec0b P 4ed7ae7b411c4a018b9e50d6c2a0104c: Deleting consensus metadata
I20260924 23:10:53.738011 14841 catalog_manager.cc:4841] TS 4e2623909988482fb519c0e8400fb98a (127.12.212.1:33967): tablet 36441591f5f44db6b2ee687a0e3cec0b (table test-table [id=56ee6b52dae4477b86ed4355420b44f9]) successfully deleted
I20260924 23:10:53.738260 14840 catalog_manager.cc:4841] TS 4ed7ae7b411c4a018b9e50d6c2a0104c (127.12.212.3:38479): tablet 36441591f5f44db6b2ee687a0e3cec0b (table test-table [id=56ee6b52dae4477b86ed4355420b44f9]) successfully deleted
I20260924 23:10:53.738837 15440 tablet_replica.cc:330] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50: stopping tablet replica
I20260924 23:10:53.738907 15440 raft_consensus.cc:2229] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:10:53.739015 15440 raft_consensus.cc:2258] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:10:53.739285 15440 ts_tablet_manager.cc:1905] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:10:53.740552 15440 ts_tablet_manager.cc:1918] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 23:10:53.740612 15440 log.cc:1196] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/36441591f5f44db6b2ee687a0e3cec0b
I20260924 23:10:53.740854 15440 ts_tablet_manager.cc:1939] T 36441591f5f44db6b2ee687a0e3cec0b P 89f6cadda86249febfd51c9858317d50: Deleting consensus metadata
I20260924 23:10:53.741292 14840 catalog_manager.cc:4841] TS 89f6cadda86249febfd51c9858317d50 (127.12.212.2:46637): tablet 36441591f5f44db6b2ee687a0e3cec0b (table test-table [id=56ee6b52dae4477b86ed4355420b44f9]) successfully deleted
Sep 24 23:10:53 dist-test-slave-m0z6 krb5kdc[14811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291453, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 23:10:53 dist-test-slave-m0z6 krb5kdc[14811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291453, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:53.764626 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:53.757530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55718 (local address 127.12.212.62:39035)
0924 23:10:53.757671 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:53.757673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:53.757683 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:10:53.757711 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:53.757712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:53.757723 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:53.757782 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:53.758388 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:53.759021 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:53.759824 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:53.760028 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:53.760083 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:53.762284 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:53.762285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:53.762286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:53.762294 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:53.763615 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:53.763917 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:53.763918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:53.763919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:53.763971 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:53.764134 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:53.764135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:53.764135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:53.764246 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:53.764337 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:53.764435 (+ 98us) server_negotiation.cc:299] Negotiation successful
0924 23:10:53.764517 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260924 23:10:53.766223 14855 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:10:53.766396 14855 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:10:53.767803 14993 tablet_service.cc:1467] Processing CreateTablet for tablet 193a7055a74940bfbd49552a603badd7 (DEFAULT_TABLE table=test-table [id=223652ebe3e846108bcc67b122c6d89c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:53.767827 15133 tablet_service.cc:1467] Processing CreateTablet for tablet 193a7055a74940bfbd49552a603badd7 (DEFAULT_TABLE table=test-table [id=223652ebe3e846108bcc67b122c6d89c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:53.767938 14993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 193a7055a74940bfbd49552a603badd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:53.767954 15133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 193a7055a74940bfbd49552a603badd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:53.768980 15449 tablet_bootstrap.cc:492] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50: Bootstrap starting.
I20260924 23:10:53.769019 15273 tablet_service.cc:1467] Processing CreateTablet for tablet 193a7055a74940bfbd49552a603badd7 (DEFAULT_TABLE table=test-table [id=223652ebe3e846108bcc67b122c6d89c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:53.769130 15273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 193a7055a74940bfbd49552a603badd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:53.769223 15450 tablet_bootstrap.cc:492] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a: Bootstrap starting.
I20260924 23:10:53.769380 15449 tablet_bootstrap.cc:654] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:53.769549 15450 tablet_bootstrap.cc:654] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:53.769927 15449 tablet_bootstrap.cc:492] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50: No bootstrap required, opened a new log
I20260924 23:10:53.769971 15449 ts_tablet_manager.cc:1397] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 23:10:53.770094 15450 tablet_bootstrap.cc:492] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a: No bootstrap required, opened a new log
I20260924 23:10:53.770140 15450 ts_tablet_manager.cc:1397] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 23:10:53.770100 15449 raft_consensus.cc:348] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:53.770169 15449 raft_consensus.cc:374] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:53.770191 15449 raft_consensus.cc:729] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f6cadda86249febfd51c9858317d50, State: Initialized, Role: FOLLOWER
I20260924 23:10:53.770238 15449 consensus_queue.cc:260] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:53.770330 15449 ts_tablet_manager.cc:1428] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:53.770256 15450 raft_consensus.cc:348] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:53.770536 15450 raft_consensus.cc:374] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:53.770573 15450 raft_consensus.cc:729] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e2623909988482fb519c0e8400fb98a, State: Initialized, Role: FOLLOWER
I20260924 23:10:53.770633 15450 consensus_queue.cc:260] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:53.770766 15450 ts_tablet_manager.cc:1428] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 23:10:53.770838 15453 tablet_bootstrap.cc:492] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Bootstrap starting.
I20260924 23:10:53.771229 15453 tablet_bootstrap.cc:654] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:53.771822 15453 tablet_bootstrap.cc:492] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c: No bootstrap required, opened a new log
I20260924 23:10:53.771867 15453 ts_tablet_manager.cc:1397] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 23:10:53.772018 15453 raft_consensus.cc:348] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:53.772076 15453 raft_consensus.cc:374] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:53.772089 15453 raft_consensus.cc:729] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed7ae7b411c4a018b9e50d6c2a0104c, State: Initialized, Role: FOLLOWER
I20260924 23:10:53.772135 15453 consensus_queue.cc:260] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:53.772282 15453 ts_tablet_manager.cc:1428] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 23:10:53.858140 15425 raft_consensus.cc:482] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:10:53.858237 15425 raft_consensus.cc:504] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:53.858400 15425 leader_election.cc:288] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89f6cadda86249febfd51c9858317d50 (127.12.212.2:46637), 4ed7ae7b411c4a018b9e50d6c2a0104c (127.12.212.3:38479)
I20260924 23:10:53.858666 15293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "193a7055a74940bfbd49552a603badd7" candidate_uuid: "4e2623909988482fb519c0e8400fb98a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" is_pre_election: true
I20260924 23:10:53.858642 15153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "193a7055a74940bfbd49552a603badd7" candidate_uuid: "4e2623909988482fb519c0e8400fb98a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f6cadda86249febfd51c9858317d50" is_pre_election: true
I20260924 23:10:53.858778 15293 raft_consensus.cc:2454] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e2623909988482fb519c0e8400fb98a in term 0.
I20260924 23:10:53.858783 15153 raft_consensus.cc:2454] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e2623909988482fb519c0e8400fb98a in term 0.
I20260924 23:10:53.858934 14944 leader_election.cc:302] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e2623909988482fb519c0e8400fb98a, 4ed7ae7b411c4a018b9e50d6c2a0104c; no voters:
I20260924 23:10:53.859045 15425 raft_consensus.cc:2781] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:10:53.859093 15425 raft_consensus.cc:482] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:10:53.859113 15425 raft_consensus.cc:3037] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:53.859907 15425 raft_consensus.cc:504] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:53.860044 15425 leader_election.cc:288] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [CANDIDATE]: Term 1 election: Requested vote from peers 89f6cadda86249febfd51c9858317d50 (127.12.212.2:46637), 4ed7ae7b411c4a018b9e50d6c2a0104c (127.12.212.3:38479)
I20260924 23:10:53.860285 15293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "193a7055a74940bfbd49552a603badd7" candidate_uuid: "4e2623909988482fb519c0e8400fb98a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c"
I20260924 23:10:53.860281 15153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "193a7055a74940bfbd49552a603badd7" candidate_uuid: "4e2623909988482fb519c0e8400fb98a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f6cadda86249febfd51c9858317d50"
I20260924 23:10:53.860361 15293 raft_consensus.cc:3037] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:53.860366 15153 raft_consensus.cc:3037] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:53.860942 15293 raft_consensus.cc:2454] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e2623909988482fb519c0e8400fb98a in term 1.
I20260924 23:10:53.860944 15153 raft_consensus.cc:2454] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e2623909988482fb519c0e8400fb98a in term 1.
I20260924 23:10:53.861135 14944 leader_election.cc:302] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e2623909988482fb519c0e8400fb98a, 4ed7ae7b411c4a018b9e50d6c2a0104c; no voters:
I20260924 23:10:53.861228 15425 raft_consensus.cc:2781] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:10:53.861289 15425 raft_consensus.cc:686] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [term 1 LEADER]: Becoming Leader. State: Replica: 4e2623909988482fb519c0e8400fb98a, State: Running, Role: LEADER
I20260924 23:10:53.861339 15425 consensus_queue.cc:237] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:53.861959 14855 catalog_manager.cc:5495] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e2623909988482fb519c0e8400fb98a (127.12.212.1). New cstate: current_term: 1 leader_uuid: "4e2623909988482fb519c0e8400fb98a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } health_report { overall_health: UNKNOWN } } }
I20260924 23:10:53.892995 15458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:53.890421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34150 (local address 127.12.212.1:33967)
0924 23:10:53.890637 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:53.890639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:53.890648 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:10:53.890668 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:53.890669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:53.890678 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:53.890762 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:10:53.890936 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:53.891435 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:53.892250 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:53.892458 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:53.892527 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:53.892713 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:10:53.892748 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:10:53.892801 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:53.892904 (+ 103us) server_negotiation.cc:299] Negotiation successful
0924 23:10:53.892926 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":113,"threads_started":1}
I20260924 23:10:53.898154 15458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:53.895698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:38039 (local address 127.12.212.1:33967)
0924 23:10:53.895743 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:53.895745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:53.895768 (+ 23us) server_negotiation.cc:407] Connection header received
0924 23:10:53.895825 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:53.895825 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:53.895835 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:53.895883 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:10:53.896093 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:53.896561 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:53.897351 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:53.897592 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:53.897652 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:53.897832 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:10:53.897878 (+ 46us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:10:53.897931 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:53.898021 (+ 90us) server_negotiation.cc:299] Negotiation successful
0924 23:10:53.898053 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":26}
I20260924 23:10:53.898252 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:53.895599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:33967 (local address 127.12.212.1:38039)
0924 23:10:53.895719 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:53.895729 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:53.895780 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:53.895919 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:53.895920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:53.895922 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 23:10:53.895995 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:53.895997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:53.896642 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:53.896643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:53.897265 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:53.897267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:53.897693 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:53.897694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:53.897752 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:53.897757 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 23:10:53.897968 (+ 211us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 23:10:53.897971 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:10:53.898016 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 23:10:53.898084 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":14,"thread_start_us":62,"threads_started":1}
I20260924 23:10:53.900175 15153 raft_consensus.cc:1261] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50 [term 1 FOLLOWER]: Refusing update from remote peer 4e2623909988482fb519c0e8400fb98a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:10:53.900218 15293 raft_consensus.cc:1261] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 1 FOLLOWER]: Refusing update from remote peer 4e2623909988482fb519c0e8400fb98a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:10:53.900413 15420 consensus_queue.cc:1035] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [LEADER]: Connected to new peer: Peer: permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:10:53.900512 15368 consensus_queue.cc:1035] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:10:53.911311 14855 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:10:53.911417 14855 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:10:53.912007 14855 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71: Sending DeleteTablet for 3 replicas of tablet 193a7055a74940bfbd49552a603badd7
I20260924 23:10:53.912228 14993 tablet_service.cc:1514] Processing DeleteTablet for tablet 193a7055a74940bfbd49552a603badd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:10:53 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:34132
I20260924 23:10:53.912246 15133 tablet_service.cc:1514] Processing DeleteTablet for tablet 193a7055a74940bfbd49552a603badd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:10:53 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:45282
I20260924 23:10:53.912319 15439 tablet_replica.cc:330] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a: stopping tablet replica
I20260924 23:10:53.912354 15440 tablet_replica.cc:330] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50: stopping tablet replica
I20260924 23:10:53.912369 15439 raft_consensus.cc:2229] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:10:53.912351 15273 tablet_service.cc:1514] Processing DeleteTablet for tablet 193a7055a74940bfbd49552a603badd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:10:53 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:53366
I20260924 23:10:53.912406 15440 raft_consensus.cc:2229] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:10:53.912441 15440 raft_consensus.cc:2258] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:10:53.912464 15439 raft_consensus.cc:2258] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:10:53.912459 15438 tablet_replica.cc:330] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c: stopping tablet replica
I20260924 23:10:53.912514 15438 raft_consensus.cc:2229] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:10:53.912550 15438 raft_consensus.cc:2258] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:10:53.912948 15438 ts_tablet_manager.cc:1905] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:10:53.914173 15439 ts_tablet_manager.cc:1905] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:10:53.914361 15438 ts_tablet_manager.cc:1918] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 23:10:53.914423 15438 log.cc:1196] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/193a7055a74940bfbd49552a603badd7
I20260924 23:10:53.914642 15438 ts_tablet_manager.cc:1939] T 193a7055a74940bfbd49552a603badd7 P 4ed7ae7b411c4a018b9e50d6c2a0104c: Deleting consensus metadata
I20260924 23:10:53.915061 14840 catalog_manager.cc:4841] TS 4ed7ae7b411c4a018b9e50d6c2a0104c (127.12.212.3:38479): tablet 193a7055a74940bfbd49552a603badd7 (table test-table [id=223652ebe3e846108bcc67b122c6d89c]) successfully deleted
I20260924 23:10:53.915517 15439 ts_tablet_manager.cc:1918] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 23:10:53.915568 15439 log.cc:1196] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-0/wal/wals/193a7055a74940bfbd49552a603badd7
I20260924 23:10:53.915745 15439 ts_tablet_manager.cc:1939] T 193a7055a74940bfbd49552a603badd7 P 4e2623909988482fb519c0e8400fb98a: Deleting consensus metadata
I20260924 23:10:53.915907 14855 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:10:53.916059 14855 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:10:53.916256 14841 catalog_manager.cc:4841] TS 4e2623909988482fb519c0e8400fb98a (127.12.212.1:33967): tablet 193a7055a74940bfbd49552a603badd7 (table test-table [id=223652ebe3e846108bcc67b122c6d89c]) successfully deleted
I20260924 23:10:53.917284 14993 tablet_service.cc:1467] Processing CreateTablet for tablet 164813fda1254f318651e716e2d791bf (DEFAULT_TABLE table=test-table [id=1c65fd4c2c7347a78775a3a3e8d20a20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:53.917356 15133 tablet_service.cc:1467] Processing CreateTablet for tablet 164813fda1254f318651e716e2d791bf (DEFAULT_TABLE table=test-table [id=1c65fd4c2c7347a78775a3a3e8d20a20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:53.917397 14993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 164813fda1254f318651e716e2d791bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:53.917464 15133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 164813fda1254f318651e716e2d791bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:53.918597 15450 tablet_bootstrap.cc:492] T 164813fda1254f318651e716e2d791bf P 4e2623909988482fb519c0e8400fb98a: Bootstrap starting.
I20260924 23:10:53.918625 15273 tablet_service.cc:1467] Processing CreateTablet for tablet 164813fda1254f318651e716e2d791bf (DEFAULT_TABLE table=test-table [id=1c65fd4c2c7347a78775a3a3e8d20a20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:10:53.918753 15273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 164813fda1254f318651e716e2d791bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:53.918959 15450 tablet_bootstrap.cc:654] T 164813fda1254f318651e716e2d791bf P 4e2623909988482fb519c0e8400fb98a: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:53.919512 15450 tablet_bootstrap.cc:492] T 164813fda1254f318651e716e2d791bf P 4e2623909988482fb519c0e8400fb98a: No bootstrap required, opened a new log
I20260924 23:10:53.919564 15450 ts_tablet_manager.cc:1397] T 164813fda1254f318651e716e2d791bf P 4e2623909988482fb519c0e8400fb98a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 23:10:53.919679 15450 raft_consensus.cc:348] T 164813fda1254f318651e716e2d791bf P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:53.919751 15450 raft_consensus.cc:374] T 164813fda1254f318651e716e2d791bf P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:53.919770 15450 raft_consensus.cc:729] T 164813fda1254f318651e716e2d791bf P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e2623909988482fb519c0e8400fb98a, State: Initialized, Role: FOLLOWER
I20260924 23:10:53.919816 15450 consensus_queue.cc:260] T 164813fda1254f318651e716e2d791bf P 4e2623909988482fb519c0e8400fb98a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:53.919833 15449 tablet_bootstrap.cc:492] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50: Bootstrap starting.
I20260924 23:10:53.919902 15450 ts_tablet_manager.cc:1428] T 164813fda1254f318651e716e2d791bf P 4e2623909988482fb519c0e8400fb98a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 23:10:53.920135 15449 tablet_bootstrap.cc:654] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:53.920462 15453 tablet_bootstrap.cc:492] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c: Bootstrap starting.
I20260924 23:10:53.920857 15453 tablet_bootstrap.cc:654] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:53.921432 15453 tablet_bootstrap.cc:492] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c: No bootstrap required, opened a new log
I20260924 23:10:53.921484 15453 ts_tablet_manager.cc:1397] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 23:10:53.921620 15449 tablet_bootstrap.cc:492] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50: No bootstrap required, opened a new log
I20260924 23:10:53.921627 15453 raft_consensus.cc:348] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:53.921670 15449 ts_tablet_manager.cc:1397] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:10:53.921697 15453 raft_consensus.cc:374] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:53.921720 15453 raft_consensus.cc:729] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed7ae7b411c4a018b9e50d6c2a0104c, State: Initialized, Role: FOLLOWER
I20260924 23:10:53.921758 15453 consensus_queue.cc:260] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:53.921782 15449 raft_consensus.cc:348] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:53.921844 15453 ts_tablet_manager.cc:1428] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:53.921849 15449 raft_consensus.cc:374] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:53.921867 15449 raft_consensus.cc:729] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f6cadda86249febfd51c9858317d50, State: Initialized, Role: FOLLOWER
I20260924 23:10:53.921904 15449 consensus_queue.cc:260] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:53.921986 15449 ts_tablet_manager.cc:1428] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:53.922897 15440 ts_tablet_manager.cc:1905] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:10:53.924365 15440 ts_tablet_manager.cc:1918] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 23:10:53.924427 15440 log.cc:1196] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/193a7055a74940bfbd49552a603badd7
I20260924 23:10:53.924659 15440 ts_tablet_manager.cc:1939] T 193a7055a74940bfbd49552a603badd7 P 89f6cadda86249febfd51c9858317d50: Deleting consensus metadata
I20260924 23:10:53.924968 14840 catalog_manager.cc:4841] TS 89f6cadda86249febfd51c9858317d50 (127.12.212.2:46637): tablet 193a7055a74940bfbd49552a603badd7 (table test-table [id=223652ebe3e846108bcc67b122c6d89c]) successfully deleted
I20260924 23:10:53.943845 15417 raft_consensus.cc:482] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:10:53.943948 15417 raft_consensus.cc:504] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:53.944113 15417 leader_election.cc:288] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e2623909988482fb519c0e8400fb98a (127.12.212.1:33967), 4ed7ae7b411c4a018b9e50d6c2a0104c (127.12.212.3:38479)
I20260924 23:10:53.944294 15013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "164813fda1254f318651e716e2d791bf" candidate_uuid: "89f6cadda86249febfd51c9858317d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e2623909988482fb519c0e8400fb98a" is_pre_election: true
I20260924 23:10:53.944336 15293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "164813fda1254f318651e716e2d791bf" candidate_uuid: "89f6cadda86249febfd51c9858317d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" is_pre_election: true
I20260924 23:10:53.944407 15013 raft_consensus.cc:2454] T 164813fda1254f318651e716e2d791bf P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89f6cadda86249febfd51c9858317d50 in term 0.
I20260924 23:10:53.944433 15293 raft_consensus.cc:2454] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89f6cadda86249febfd51c9858317d50 in term 0.
I20260924 23:10:53.944588 15084 leader_election.cc:302] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed7ae7b411c4a018b9e50d6c2a0104c, 89f6cadda86249febfd51c9858317d50; no voters:
I20260924 23:10:53.944715 15417 raft_consensus.cc:2781] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:10:53.944746 15417 raft_consensus.cc:482] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:10:53.944766 15417 raft_consensus.cc:3037] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:53.945669 15417 raft_consensus.cc:504] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:53.945822 15417 leader_election.cc:288] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [CANDIDATE]: Term 1 election: Requested vote from peers 4e2623909988482fb519c0e8400fb98a (127.12.212.1:33967), 4ed7ae7b411c4a018b9e50d6c2a0104c (127.12.212.3:38479)
I20260924 23:10:53.945999 15293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "164813fda1254f318651e716e2d791bf" candidate_uuid: "89f6cadda86249febfd51c9858317d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c"
I20260924 23:10:53.946014 15013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "164813fda1254f318651e716e2d791bf" candidate_uuid: "89f6cadda86249febfd51c9858317d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e2623909988482fb519c0e8400fb98a"
I20260924 23:10:53.946096 15013 raft_consensus.cc:3037] T 164813fda1254f318651e716e2d791bf P 4e2623909988482fb519c0e8400fb98a [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:53.946118 15293 raft_consensus.cc:3037] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:53.946915 15013 raft_consensus.cc:2454] T 164813fda1254f318651e716e2d791bf P 4e2623909988482fb519c0e8400fb98a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89f6cadda86249febfd51c9858317d50 in term 1.
I20260924 23:10:53.946957 15293 raft_consensus.cc:2454] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89f6cadda86249febfd51c9858317d50 in term 1.
I20260924 23:10:53.947184 15085 leader_election.cc:302] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e2623909988482fb519c0e8400fb98a, 89f6cadda86249febfd51c9858317d50; no voters:
I20260924 23:10:53.947453 15417 raft_consensus.cc:2781] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:10:53.947507 15417 raft_consensus.cc:686] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [term 1 LEADER]: Becoming Leader. State: Replica: 89f6cadda86249febfd51c9858317d50, State: Running, Role: LEADER
I20260924 23:10:53.947554 15417 consensus_queue.cc:237] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } }
I20260924 23:10:53.948248 14855 catalog_manager.cc:5495] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89f6cadda86249febfd51c9858317d50 (127.12.212.2). New cstate: current_term: 1 leader_uuid: "89f6cadda86249febfd51c9858317d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89f6cadda86249febfd51c9858317d50" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 } health_report { overall_health: UNKNOWN } } }
I20260924 23:10:53.962042 15470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:53.959893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45298 (local address 127.12.212.2:46637)
0924 23:10:53.960062 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:53.960064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:53.960069 (+ 5us) server_negotiation.cc:407] Connection header received
0924 23:10:53.960086 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:53.960087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:53.960095 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:53.960148 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:10:53.960258 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:53.960738 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:53.961425 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:53.961629 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:53.961682 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:53.961801 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:10:53.961836 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:10:53.961875 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:53.961932 (+ 57us) server_negotiation.cc:299] Negotiation successful
0924 23:10:53.961954 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":106,"threads_started":1}
I20260924 23:10:53.964141 15293 raft_consensus.cc:1261] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 1 FOLLOWER]: Refusing update from remote peer 89f6cadda86249febfd51c9858317d50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:10:53.964313 15013 raft_consensus.cc:1261] T 164813fda1254f318651e716e2d791bf P 4e2623909988482fb519c0e8400fb98a [term 1 FOLLOWER]: Refusing update from remote peer 89f6cadda86249febfd51c9858317d50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:10:53.964371 15430 consensus_queue.cc:1035] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed7ae7b411c4a018b9e50d6c2a0104c" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 38479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:10:53.964622 15430 consensus_queue.cc:1035] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e2623909988482fb519c0e8400fb98a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 23:10:53.969564 15422 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260924 23:10:55.486665 14855 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:10:55.486780 14855 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:10:55.487506 14855 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40524990b4494484920a2b5f044b3a71: Sending DeleteTablet for 3 replicas of tablet 164813fda1254f318651e716e2d791bf
I20260924 23:10:55.487775 15133 tablet_service.cc:1514] Processing DeleteTablet for tablet 164813fda1254f318651e716e2d791bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:10:55 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:45282
I20260924 23:10:55.487782 15273 tablet_service.cc:1514] Processing DeleteTablet for tablet 164813fda1254f318651e716e2d791bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:10:55 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:53366
I20260924 23:10:55.487898 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 14928
I20260924 23:10:55.487958 14993 tablet_service.cc:1514] Processing DeleteTablet for tablet 164813fda1254f318651e716e2d791bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:10:55 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:34132
I20260924 23:10:55.488497 15497 tablet_replica.cc:330] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50: stopping tablet replica
I20260924 23:10:55.488557 15497 raft_consensus.cc:2229] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:10:55.488605 15497 raft_consensus.cc:2258] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:10:55.488765 15498 tablet_replica.cc:330] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c: stopping tablet replica
I20260924 23:10:55.488817 15498 raft_consensus.cc:2229] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:10:55.488822 15497 ts_tablet_manager.cc:1905] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:10:55.488895 15498 raft_consensus.cc:2258] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:10:55.489110 15498 ts_tablet_manager.cc:1905] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:10:55.490483 15497 ts_tablet_manager.cc:1918] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 23:10:55.490516 15498 ts_tablet_manager.cc:1918] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 23:10:55.490541 15497 log.cc:1196] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/164813fda1254f318651e716e2d791bf
I20260924 23:10:55.490562 15498 log.cc:1196] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/164813fda1254f318651e716e2d791bf
I20260924 23:10:55.490809 15498 ts_tablet_manager.cc:1939] T 164813fda1254f318651e716e2d791bf P 4ed7ae7b411c4a018b9e50d6c2a0104c: Deleting consensus metadata
I20260924 23:10:55.490818 15497 ts_tablet_manager.cc:1939] T 164813fda1254f318651e716e2d791bf P 89f6cadda86249febfd51c9858317d50: Deleting consensus metadata
I20260924 23:10:55.491109 14840 catalog_manager.cc:4841] TS 89f6cadda86249febfd51c9858317d50 (127.12.212.2:46637): tablet 164813fda1254f318651e716e2d791bf (table test-table [id=1c65fd4c2c7347a78775a3a3e8d20a20]) successfully deleted
I20260924 23:10:55.491182 14840 catalog_manager.cc:4841] TS 4ed7ae7b411c4a018b9e50d6c2a0104c (127.12.212.3:38479): tablet 164813fda1254f318651e716e2d791bf (table test-table [id=1c65fd4c2c7347a78775a3a3e8d20a20]) successfully deleted
W20260924 23:10:55.494923 14841 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.212.1:33967 (error 108)
I20260924 23:10:55.495146 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 15068
I20260924 23:10:55.498155 15500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:55.495304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:33967 (local address 127.0.0.1:34152)
0924 23:10:55.498041 (+ 2737us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:55.498110 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.1:33967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2703,"thread_start_us":68,"threads_started":1}
W20260924 23:10:55.498327 14841 catalog_manager.cc:4584] TS 4e2623909988482fb519c0e8400fb98a (127.12.212.1:33967): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 164813fda1254f318651e716e2d791bf: Network error: Client connection negotiation failed: client connection to 127.12.212.1:33967: connect: Connection refused (error 111)
I20260924 23:10:55.500576 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 15208
I20260924 23:10:55.505141 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 14824
2026-09-24T23:10:55Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5478 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:10:55 dist-test-slave-m0z6 krb5kdc[15503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:10:55 dist-test-slave-m0z6 krb5kdc[15503](info): set up 2 sockets
Sep 24 23:10:55 dist-test-slave-m0z6 krb5kdc[15503](info): commencing operation
krb5kdc: starting...
W20260924 23:10:56.482292 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.956s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:10:56 dist-test-slave-m0z6 krb5kdc[15503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:10:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:10:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:10:56.597865 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:39451
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:39451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:56.678917 15516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:56.679152 15516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:56.679184 15516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:56.680435 15516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:10:56.680488 15516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:56.680506 15516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:56.680524 15516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:10:56.680539 15516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:10:56.682080 15516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:39451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:39451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:56.682337 15516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:56.682555 15516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:56.687228 15524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:56.687222 15522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:56.687330 15521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:56.687388 15516 server_base.cc:1034] running on GCE node
I20260924 23:10:56.687747 15516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:56.688028 15516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:56.689173 15516 hybrid_clock.cc:648] HybridClock initialized: now 1790291456689139 us; error 47 us; skew 500 ppm
Sep 24 23:10:56 dist-test-slave-m0z6 krb5kdc[15503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291456, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:56.691669 15516 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:10:56.692133 15516 webserver.cc:460] Webserver started at http://127.12.212.62:39941/ using document root <none> and password file <none>
I20260924 23:10:56.692354 15516 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:56.692402 15516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:56.692526 15516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:56.693569 15516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "bb9d7ec581044015bd1085e923b606b3"
format_stamp: "Formatted at 2026-09-24 23:10:56 on dist-test-slave-m0z6"
server_key: "e812dbad97d828c5b42b403efdd8690e"
server_key_iv: "3275e41c28dbc39184be838553d17d43"
server_key_version: "encryptionkey@0"
I20260924 23:10:56.693903 15516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "bb9d7ec581044015bd1085e923b606b3"
format_stamp: "Formatted at 2026-09-24 23:10:56 on dist-test-slave-m0z6"
server_key: "e812dbad97d828c5b42b403efdd8690e"
server_key_iv: "3275e41c28dbc39184be838553d17d43"
server_key_version: "encryptionkey@0"
I20260924 23:10:56.695216 15516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:10:56.695971 15531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:56.696130 15516 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 23:10:56.696204 15516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "bb9d7ec581044015bd1085e923b606b3"
format_stamp: "Formatted at 2026-09-24 23:10:56 on dist-test-slave-m0z6"
server_key: "e812dbad97d828c5b42b403efdd8690e"
server_key_iv: "3275e41c28dbc39184be838553d17d43"
server_key_version: "encryptionkey@0"
I20260924 23:10:56.696269 15516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:56.712141 15516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:56.714310 15516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:56.714504 15516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:56.718381 15583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:39451 every 8 connection(s)
I20260924 23:10:56.718382 15516 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:39451
I20260924 23:10:56.718870 15516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:10:56.719727 15584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:56.721954 15584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3: Bootstrap starting.
I20260924 23:10:56.722482 15584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:56.722798 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 15516
I20260924 23:10:56.722824 15584 log.cc:824] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3: Log is configured to *not* fsync() on all Append() calls
I20260924 23:10:56.722904 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:10:56.723116 13136 external_mini_cluster.cc:1428] Setting key c238f187bdf202ef9e016a14d7f24324
I20260924 23:10:56.723624 15584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3: No bootstrap required, opened a new log
I20260924 23:10:56.726519 15584 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9d7ec581044015bd1085e923b606b3" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39451 } }
I20260924 23:10:56.726752 15584 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:56.726800 15584 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb9d7ec581044015bd1085e923b606b3, State: Initialized, Role: FOLLOWER
I20260924 23:10:56.726935 15584 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9d7ec581044015bd1085e923b606b3" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39451 } }
I20260924 23:10:56.727010 15584 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:10:56.727043 15584 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:10:56.727085 15584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:56.727733 15584 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9d7ec581044015bd1085e923b606b3" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39451 } }
I20260924 23:10:56.727837 15584 leader_election.cc:302] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb9d7ec581044015bd1085e923b606b3; no voters:
I20260924 23:10:56.728057 15584 leader_election.cc:288] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:10:56.728076 15589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:10:56.728276 15589 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3 [term 1 LEADER]: Becoming Leader. State: Replica: bb9d7ec581044015bd1085e923b606b3, State: Running, Role: LEADER
I20260924 23:10:56.728356 15584 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:10:56.728394 15589 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9d7ec581044015bd1085e923b606b3" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39451 } }
I20260924 23:10:56.728724 15589 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb9d7ec581044015bd1085e923b606b3. Latest consensus state: current_term: 1 leader_uuid: "bb9d7ec581044015bd1085e923b606b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9d7ec581044015bd1085e923b606b3" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39451 } } }
I20260924 23:10:56.728876 15589 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3 [sys.catalog]: This master's current role is: LEADER
I20260924 23:10:56.729111 15598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:10:56.729803 15598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 23:10:56 dist-test-slave-m0z6 krb5kdc[15503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:56.730948 15591 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb9d7ec581044015bd1085e923b606b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9d7ec581044015bd1085e923b606b3" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39451 } } }
I20260924 23:10:56.731094 15591 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3 [sys.catalog]: This master's current role is: LEADER
I20260924 23:10:56.734133 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:56.723635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43688 (local address 127.12.212.62:39451)
0924 23:10:56.723778 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:56.723788 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:56.723815 (+ 27us) server_negotiation.cc:407] Connection header received
0924 23:10:56.724344 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:56.724353 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:56.724407 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:56.724532 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:56.725333 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:56.726266 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:56.727321 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:56.727754 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:56.727890 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:56.731000 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:56.731005 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:56.731005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:56.731019 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:56.732655 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:56.733005 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:56.733008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:56.733008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:56.733099 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:56.733280 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:56.733281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:56.733281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:56.733539 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:56.733706 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:56.733794 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 23:10:56.733938 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":66,"threads_started":1}
I20260924 23:10:56.735193 15598 catalog_manager.cc:1347] Generated new cluster ID: 9582f7b8a190480188d90e2aa8f7fd1b
I20260924 23:10:56.735260 15598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:10:56.748508 15598 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:10:56.749409 15598 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:10:56.763353 15598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb9d7ec581044015bd1085e923b606b3: Generated new TSK 0
I20260924 23:10:56.763616 15598 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
I20260924 23:10:56.793689 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:39451
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:56.875442 15610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:56.875648 15610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:56.875679 15610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:56.876804 15610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:56.876840 15610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:56.876864 15610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:10:56.878273 15610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:39451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:56.878464 15610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:56.878646 15610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:56.883517 15618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:56.883513 15615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:56.883746 15610 server_base.cc:1034] running on GCE node
W20260924 23:10:56.883518 15616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:56.884111 15610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:56.884378 15610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:56.885522 15610 hybrid_clock.cc:648] HybridClock initialized: now 1790291456885506 us; error 27 us; skew 500 ppm
Sep 24 23:10:56 dist-test-slave-m0z6 krb5kdc[15503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291456, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:56.888111 15610 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:10:56.888550 15610 webserver.cc:460] Webserver started at http://127.12.212.1:46363/ using document root <none> and password file <none>
I20260924 23:10:56.888728 15610 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:56.888767 15610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:56.888864 15610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:56.889909 15610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "65fd849549dd4a56a0b334b8d544f3e3"
format_stamp: "Formatted at 2026-09-24 23:10:56 on dist-test-slave-m0z6"
server_key: "a8051b90fe97c7fc7982b2ae6b2f809c"
server_key_iv: "8dccec9ecc7d08064da521aa858a2f44"
server_key_version: "encryptionkey@0"
I20260924 23:10:56.890209 15610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "65fd849549dd4a56a0b334b8d544f3e3"
format_stamp: "Formatted at 2026-09-24 23:10:56 on dist-test-slave-m0z6"
server_key: "a8051b90fe97c7fc7982b2ae6b2f809c"
server_key_iv: "8dccec9ecc7d08064da521aa858a2f44"
server_key_version: "encryptionkey@0"
I20260924 23:10:56.891625 15610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260924 23:10:56.892705 15625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:56.892932 15610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:10:56.893023 15610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "65fd849549dd4a56a0b334b8d544f3e3"
format_stamp: "Formatted at 2026-09-24 23:10:56 on dist-test-slave-m0z6"
server_key: "a8051b90fe97c7fc7982b2ae6b2f809c"
server_key_iv: "8dccec9ecc7d08064da521aa858a2f44"
server_key_version: "encryptionkey@0"
I20260924 23:10:56.893090 15610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:56.922775 15610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:56.925083 15610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:56.925204 15610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:56.925402 15610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:10:56.925709 15610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:10:56.925760 15610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:56.925791 15610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:10:56.925808 15610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:56.931586 15610 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:45895
I20260924 23:10:56.931655 15738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:45895 every 8 connection(s)
I20260924 23:10:56.931963 15610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
Sep 24 23:10:56 dist-test-slave-m0z6 krb5kdc[15503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291456, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:56.938846 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 15610
I20260924 23:10:56.938942 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:10:56.939127 13136 external_mini_cluster.cc:1428] Setting key 822f31bad4bdedd653a898844105aab6
I20260924 23:10:56.941558 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:56.932691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:35527 (local address 127.12.212.62:39451)
0924 23:10:56.932745 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:56.932748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:56.932952 (+ 204us) server_negotiation.cc:407] Connection header received
0924 23:10:56.933468 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:56.933469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:56.933482 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:56.933534 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:56.934552 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:56.935102 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:56.936072 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:56.936273 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:56.936330 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:56.938427 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:56.938428 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:56.938429 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:56.938437 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:56.940018 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:56.940520 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:56.940521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:56.940522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:56.940586 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:56.940840 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:56.940840 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:56.940841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:56.940986 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:56.941054 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:56.941406 (+ 352us) server_negotiation.cc:299] Negotiation successful
0924 23:10:56.941490 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 23:10:56.941817 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:56.932653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:39451 (local address 127.12.212.1:35527)
0924 23:10:56.932827 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:56.932865 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:56.933364 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:56.933611 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:56.933615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:56.934184 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:56.934441 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:56.934447 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:56.935187 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:56.935188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:56.935986 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:56.935988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:56.936343 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:56.936344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:56.936470 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:56.936966 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:56.936975 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:56.938330 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:56.940147 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:56.940158 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:56.940160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:56.940444 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:56.940645 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:56.940645 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:56.940645 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:56.940770 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:56.941052 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:56.941059 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:56.941251 (+ 192us) client_negotiation.cc:769] Sending connection context
0924 23:10:56.941374 (+ 123us) client_negotiation.cc:240] Negotiation successful
0924 23:10:56.941564 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260924 23:10:56.942379 15739 heartbeater.cc:347] Connected to a master server at 127.12.212.62:39451
I20260924 23:10:56.942472 15739 heartbeater.cc:464] Registering TS with master...
I20260924 23:10:56.942651 15739 heartbeater.cc:511] Master 127.12.212.62:39451 requested a full tablet report, sending...
I20260924 23:10:56.943198 15548 ts_manager.cc:194] Registered new tserver with Master: 65fd849549dd4a56a0b334b8d544f3e3 (127.12.212.1:45895)
I20260924 23:10:56.943831 15548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:35527
WARNING: no policy specified for kudu/127.12.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
I20260924 23:10:56.965862 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:39451
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:57.046409 15744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:57.046638 15744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:57.046662 15744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:57.047924 15744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:57.047976 15744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:57.048003 15744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:10:57.049533 15744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:39451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:57.049779 15744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:57.049993 15744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:57.054798 15749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:57.054955 15750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:57.054965 15752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:57.055076 15744 server_base.cc:1034] running on GCE node
I20260924 23:10:57.055258 15744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:57.055480 15744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:57.056612 15744 hybrid_clock.cc:648] HybridClock initialized: now 1790291457056592 us; error 35 us; skew 500 ppm
Sep 24 23:10:57 dist-test-slave-m0z6 krb5kdc[15503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291457, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:57.058879 15744 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:10:57.059288 15744 webserver.cc:460] Webserver started at http://127.12.212.2:32769/ using document root <none> and password file <none>
I20260924 23:10:57.059466 15744 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:57.059505 15744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:57.059593 15744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:57.060549 15744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "ead4be0cd11449f1a8fbd853b8c4175b"
format_stamp: "Formatted at 2026-09-24 23:10:57 on dist-test-slave-m0z6"
server_key: "7950b52f503b616fa13adbf3ca1c5a45"
server_key_iv: "265cc8944b0e33f9abc6baecfd4f0a08"
server_key_version: "encryptionkey@0"
I20260924 23:10:57.060840 15744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "ead4be0cd11449f1a8fbd853b8c4175b"
format_stamp: "Formatted at 2026-09-24 23:10:57 on dist-test-slave-m0z6"
server_key: "7950b52f503b616fa13adbf3ca1c5a45"
server_key_iv: "265cc8944b0e33f9abc6baecfd4f0a08"
server_key_version: "encryptionkey@0"
I20260924 23:10:57.061932 15744 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 23:10:57.062529 15759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:57.062669 15744 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:57.062785 15744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "ead4be0cd11449f1a8fbd853b8c4175b"
format_stamp: "Formatted at 2026-09-24 23:10:57 on dist-test-slave-m0z6"
server_key: "7950b52f503b616fa13adbf3ca1c5a45"
server_key_iv: "265cc8944b0e33f9abc6baecfd4f0a08"
server_key_version: "encryptionkey@0"
I20260924 23:10:57.062841 15744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:57.074314 15744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:57.076407 15744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:57.076521 15744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:57.076706 15744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:10:57.076997 15744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:10:57.077036 15744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:57.077059 15744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:10:57.077081 15744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:57.082485 15744 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:38249
I20260924 23:10:57.082547 15872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:38249 every 8 connection(s)
I20260924 23:10:57.082890 15744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
Sep 24 23:10:57 dist-test-slave-m0z6 krb5kdc[15503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291457, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:57.090687 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 15744
I20260924 23:10:57.090814 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:10:57.090984 13136 external_mini_cluster.cc:1428] Setting key 537a9f057a114b458b10f1d9e036706f
I20260924 23:10:57.092233 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:57.083576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:47663 (local address 127.12.212.62:39451)
0924 23:10:57.083627 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:57.083629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:57.083842 (+ 213us) server_negotiation.cc:407] Connection header received
0924 23:10:57.084386 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:57.084388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:57.084400 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:57.084451 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:57.085393 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:57.085882 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:57.086904 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:57.087091 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:57.087148 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:57.089329 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:57.089330 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:57.089331 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:57.089339 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:57.090626 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:57.091099 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:57.091101 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:57.091101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:57.091167 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:57.091477 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:57.091477 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:57.091478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:57.091630 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:57.091711 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:57.092078 (+ 367us) server_negotiation.cc:299] Negotiation successful
0924 23:10:57.092150 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 23:10:57.092447 15875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:57.083555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:39451 (local address 127.12.212.2:47663)
0924 23:10:57.083698 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:57.083734 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:57.084280 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:57.084527 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:57.084531 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:57.085094 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:57.085294 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:57.085299 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:57.085964 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:57.085965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:57.086813 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:57.086816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:57.087160 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:57.087161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:57.087291 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:57.087807 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:57.087816 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:57.089180 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:57.090751 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:57.090760 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:57.090762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:57.091032 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:57.091261 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:57.091262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:57.091262 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:57.091408 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:57.091718 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:57.091724 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:57.091917 (+ 193us) client_negotiation.cc:769] Sending connection context
0924 23:10:57.092030 (+ 113us) client_negotiation.cc:240] Negotiation successful
0924 23:10:57.092219 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
I20260924 23:10:57.092922 15873 heartbeater.cc:347] Connected to a master server at 127.12.212.62:39451
I20260924 23:10:57.093036 15873 heartbeater.cc:464] Registering TS with master...
I20260924 23:10:57.093222 15873 heartbeater.cc:511] Master 127.12.212.62:39451 requested a full tablet report, sending...
I20260924 23:10:57.093596 15548 ts_manager.cc:194] Registered new tserver with Master: ead4be0cd11449f1a8fbd853b8c4175b (127.12.212.2:38249)
I20260924 23:10:57.094056 15548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:47663
WARNING: no policy specified for kudu/127.12.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
I20260924 23:10:57.118074 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:39451
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:57.199295 15878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:57.199529 15878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:57.199563 15878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:57.200816 15878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:57.200870 15878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:57.200908 15878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:10:57.202435 15878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:39451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:57.202716 15878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:57.202967 15878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:10:57.208014 15878 server_base.cc:1034] running on GCE node
W20260924 23:10:57.208185 15886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:57.208026 15884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:57.208019 15883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:57.208514 15878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:57.208762 15878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:57.209918 15878 hybrid_clock.cc:648] HybridClock initialized: now 1790291457209902 us; error 30 us; skew 500 ppm
Sep 24 23:10:57 dist-test-slave-m0z6 krb5kdc[15503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291457, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:57.212560 15878 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:10:57.213022 15878 webserver.cc:460] Webserver started at http://127.12.212.3:34627/ using document root <none> and password file <none>
I20260924 23:10:57.213233 15878 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:57.213280 15878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:57.213410 15878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:57.214454 15878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "548411d2b20949128367a5647cb44ef4"
format_stamp: "Formatted at 2026-09-24 23:10:57 on dist-test-slave-m0z6"
server_key: "4f07ee61914165acc579d6861ddf51ff"
server_key_iv: "cd3d35e245166ce596017353276daa7c"
server_key_version: "encryptionkey@0"
I20260924 23:10:57.214841 15878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "548411d2b20949128367a5647cb44ef4"
format_stamp: "Formatted at 2026-09-24 23:10:57 on dist-test-slave-m0z6"
server_key: "4f07ee61914165acc579d6861ddf51ff"
server_key_iv: "cd3d35e245166ce596017353276daa7c"
server_key_version: "encryptionkey@0"
I20260924 23:10:57.216116 15878 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:10:57.216785 15893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:57.216934 15878 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 23:10:57.217012 15878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "548411d2b20949128367a5647cb44ef4"
format_stamp: "Formatted at 2026-09-24 23:10:57 on dist-test-slave-m0z6"
server_key: "4f07ee61914165acc579d6861ddf51ff"
server_key_iv: "cd3d35e245166ce596017353276daa7c"
server_key_version: "encryptionkey@0"
I20260924 23:10:57.217082 15878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:57.236732 15878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:57.238834 15878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:57.238963 15878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:57.239190 15878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:10:57.239497 15878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:10:57.239529 15878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:57.239560 15878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:10:57.239583 15878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:57.245199 15878 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:36013
I20260924 23:10:57.245247 16006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:36013 every 8 connection(s)
I20260924 23:10:57.245558 15878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
Sep 24 23:10:57 dist-test-slave-m0z6 krb5kdc[15503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291457, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:57.253352 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 15878
I20260924 23:10:57.253453 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:10:57.253652 13136 external_mini_cluster.cc:1428] Setting key 652dc44bbb6b4f86ef53fcac37f57bd5
I20260924 23:10:57.254830 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:57.246204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:54625 (local address 127.12.212.62:39451)
0924 23:10:57.246264 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:57.246266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:57.246485 (+ 219us) server_negotiation.cc:407] Connection header received
0924 23:10:57.247058 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:57.247059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:57.247069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:57.247117 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:57.248012 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:57.248529 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:57.249555 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:57.249755 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:57.249814 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:57.252045 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:57.252047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:57.252047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:57.252055 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:57.253246 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:57.253733 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:57.253734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:57.253735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:57.253795 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:57.254114 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:57.254115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:57.254115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:57.254259 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:57.254331 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:57.254641 (+ 310us) server_negotiation.cc:299] Negotiation successful
0924 23:10:57.254742 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 23:10:57.255036 16009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:57.246228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:39451 (local address 127.12.212.3:54625)
0924 23:10:57.246377 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:57.246416 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:57.246975 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:57.247177 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:57.247181 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:57.247732 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:57.247940 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:57.247945 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:57.248618 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:57.248619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:57.249461 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:57.249463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:57.249839 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:57.249840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:57.249948 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:57.250435 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:57.250444 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:57.251940 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:57.253345 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:57.253353 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:57.253355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:57.253660 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:57.253881 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:57.253882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:57.253882 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:57.254040 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:57.254337 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:57.254344 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:57.254496 (+ 152us) client_negotiation.cc:769] Sending connection context
0924 23:10:57.254608 (+ 112us) client_negotiation.cc:240] Negotiation successful
0924 23:10:57.254828 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260924 23:10:57.255438 16007 heartbeater.cc:347] Connected to a master server at 127.12.212.62:39451
I20260924 23:10:57.255533 16007 heartbeater.cc:464] Registering TS with master...
I20260924 23:10:57.255708 16007 heartbeater.cc:511] Master 127.12.212.62:39451 requested a full tablet report, sending...
I20260924 23:10:57.256059 15548 ts_manager.cc:194] Registered new tserver with Master: 548411d2b20949128367a5647cb44ef4 (127.12.212.3:36013)
I20260924 23:10:57.256484 15548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:54625
I20260924 23:10:57.266070 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 23:10:57.276674 15588 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:10:57.275843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43694 (local address 127.12.212.62:39451)
0924 23:10:57.275892 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:57.275894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:57.275969 (+ 75us) server_negotiation.cc:407] Connection header received
0924 23:10:57.276021 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:57.276022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:57.276030 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:57.276089 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:57.276617 (+ 528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43694: BlockingRecv error: recv got EOF from 127.0.0.1:43694 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 23:10:57.277465 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 15610
I20260924 23:10:57.281286 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 15744
I20260924 23:10:57.285511 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 15878
I20260924 23:10:57.289309 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 15516
2026-09-24T23:10:57Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1782 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:10:57 dist-test-slave-m0z6 krb5kdc[16019](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:10:57 dist-test-slave-m0z6 krb5kdc[16019](info): set up 2 sockets
Sep 24 23:10:57 dist-test-slave-m0z6 krb5kdc[16019](info): commencing operation
krb5kdc: starting...
W20260924 23:10:58.238340 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:10:58 dist-test-slave-m0z6 krb5kdc[16019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:10:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:10:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:10:58.361624 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:46011
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:33089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:46011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:58.443897 16032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:58.444167 16032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:58.444204 16032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:58.445528 16032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:10:58.445583 16032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:58.445602 16032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:58.445618 16032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:10:58.445633 16032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:10:58.447335 16032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:33089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:46011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:46011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:58.447741 16032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:58.448010 16032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:58.453044 16038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:58.453096 16037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:58.453055 16040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:58.453102 16032 server_base.cc:1034] running on GCE node
I20260924 23:10:58.453524 16032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:58.453851 16032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:58.455026 16032 hybrid_clock.cc:648] HybridClock initialized: now 1790291458454969 us; error 74 us; skew 500 ppm
Sep 24 23:10:58 dist-test-slave-m0z6 krb5kdc[16019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291458, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:58.457855 16032 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:10:58.458374 16032 webserver.cc:460] Webserver started at http://127.12.212.62:41045/ using document root <none> and password file <none>
I20260924 23:10:58.458637 16032 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:58.458719 16032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:58.458854 16032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:58.459950 16032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "3510154c23ad4fba85d76c789e4162e0"
format_stamp: "Formatted at 2026-09-24 23:10:58 on dist-test-slave-m0z6"
server_key: "ef4f9d8a96f41fa371c5d989bd7aec5b"
server_key_iv: "2728de28f2569152efd1ab74c8a2cfee"
server_key_version: "encryptionkey@0"
I20260924 23:10:58.460301 16032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "3510154c23ad4fba85d76c789e4162e0"
format_stamp: "Formatted at 2026-09-24 23:10:58 on dist-test-slave-m0z6"
server_key: "ef4f9d8a96f41fa371c5d989bd7aec5b"
server_key_iv: "2728de28f2569152efd1ab74c8a2cfee"
server_key_version: "encryptionkey@0"
I20260924 23:10:58.461841 16032 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:10:58.462824 16047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:58.463016 16032 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:10:58.463128 16032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "3510154c23ad4fba85d76c789e4162e0"
format_stamp: "Formatted at 2026-09-24 23:10:58 on dist-test-slave-m0z6"
server_key: "ef4f9d8a96f41fa371c5d989bd7aec5b"
server_key_iv: "2728de28f2569152efd1ab74c8a2cfee"
server_key_version: "encryptionkey@0"
I20260924 23:10:58.463215 16032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:58.481351 16032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:58.484154 16032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:58.484328 16032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:58.488685 16032 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:46011
I20260924 23:10:58.488783 16099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:46011 every 8 connection(s)
I20260924 23:10:58.489322 16032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:10:58.490094 16100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:58.492708 16100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0: Bootstrap starting.
I20260924 23:10:58.493369 16100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:58.493670 16100 log.cc:824] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0: Log is configured to *not* fsync() on all Append() calls
I20260924 23:10:58.494542 16100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0: No bootstrap required, opened a new log
I20260924 23:10:58.496281 16100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3510154c23ad4fba85d76c789e4162e0" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46011 } }
I20260924 23:10:58.496464 16100 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:58.496486 16100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3510154c23ad4fba85d76c789e4162e0, State: Initialized, Role: FOLLOWER
I20260924 23:10:58.496619 16100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3510154c23ad4fba85d76c789e4162e0" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46011 } }
I20260924 23:10:58.496757 16100 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:10:58.496780 16100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:10:58.496811 16100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:58.497015 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 16032
I20260924 23:10:58.497094 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:10:58.497293 13136 external_mini_cluster.cc:1428] Setting key c565b7a0bcde35895beff3a39750c671
I20260924 23:10:58.497861 16100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3510154c23ad4fba85d76c789e4162e0" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46011 } }
I20260924 23:10:58.498019 16100 leader_election.cc:302] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3510154c23ad4fba85d76c789e4162e0; no voters:
I20260924 23:10:58.498178 16100 leader_election.cc:288] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:10:58.498214 16105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:10:58.498440 16100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:10:58.498458 16105 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0 [term 1 LEADER]: Becoming Leader. State: Replica: 3510154c23ad4fba85d76c789e4162e0, State: Running, Role: LEADER
I20260924 23:10:58.498685 16105 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3510154c23ad4fba85d76c789e4162e0" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46011 } }
I20260924 23:10:58.499126 16107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3510154c23ad4fba85d76c789e4162e0. Latest consensus state: current_term: 1 leader_uuid: "3510154c23ad4fba85d76c789e4162e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3510154c23ad4fba85d76c789e4162e0" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46011 } } }
I20260924 23:10:58.499217 16107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0 [sys.catalog]: This master's current role is: LEADER
I20260924 23:10:58.499362 16106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3510154c23ad4fba85d76c789e4162e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3510154c23ad4fba85d76c789e4162e0" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46011 } } }
I20260924 23:10:58.499428 16106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0 [sys.catalog]: This master's current role is: LEADER
I20260924 23:10:58.499405 16114 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:10:58.499931 16114 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:10:58.501521 16114 catalog_manager.cc:1347] Generated new cluster ID: 0eb7bad41a8c446e84e0019612c34405
I20260924 23:10:58.501574 16114 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 23:10:58 dist-test-slave-m0z6 krb5kdc[16019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:58.509829 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:58.497779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45450 (local address 127.12.212.62:46011)
0924 23:10:58.497992 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:58.498005 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:58.498045 (+ 40us) server_negotiation.cc:407] Connection header received
0924 23:10:58.498182 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:58.498190 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:58.498256 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:58.498447 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:58.499279 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:58.500722 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:58.501713 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:58.502363 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:58.502476 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:58.505292 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:58.505298 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:58.505298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:58.505314 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:58.508033 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:58.508477 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:58.508482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:58.508483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:58.508642 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:58.508825 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:58.508826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:58.508826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:58.509179 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:58.509348 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:58.509424 (+ 76us) server_negotiation.cc:299] Negotiation successful
0924 23:10:58.509588 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":100,"threads_started":1}
I20260924 23:10:58.510764 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:10:58.522168 16114 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:10:58.523005 16114 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:10:58.526371 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:58.523072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56780 (local address 127.12.212.62:46011)
0924 23:10:58.523146 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:58.523149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:58.523158 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:10:58.523183 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:58.523184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:58.523194 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:58.523256 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:58.523692 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:58.524203 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:58.524971 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:58.525189 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:58.525250 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:58.525923 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:58.525925 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:58.525925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:10:58.526119 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:58.526136 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:10:58.526142 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:58.526188 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:58.526255 (+ 67us) server_negotiation.cc:299] Negotiation successful
0924 23:10:58.526287 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260924 23:10:58.533138 16114 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3510154c23ad4fba85d76c789e4162e0: Generated new TSK 0
I20260924 23:10:58.533372 16114 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:10:58.539345 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 16032
2026-09-24T23:10:58Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1248 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:10:58 dist-test-slave-m0z6 krb5kdc[16133](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:10:58 dist-test-slave-m0z6 krb5kdc[16133](info): set up 2 sockets
Sep 24 23:10:58 dist-test-slave-m0z6 krb5kdc[16133](info): commencing operation
krb5kdc: starting...
W20260924 23:10:59.506419 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.950s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:10:59 dist-test-slave-m0z6 krb5kdc[16133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:10:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:10:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:10:59.626355 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:46379
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:44459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:46379
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:59.708498 16146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:59.708757 16146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:59.708791 16146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:59.710100 16146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:10:59.710156 16146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:59.710176 16146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:59.710192 16146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:10:59.710207 16146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:10:59.711874 16146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:44459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:46379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:46379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:59.712182 16146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:59.712411 16146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:10:59.717571 16146 server_base.cc:1034] running on GCE node
W20260924 23:10:59.717504 16152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:59.717523 16151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:59.717555 16154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:59.718024 16146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:59.718400 16146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:59.719605 16146 hybrid_clock.cc:648] HybridClock initialized: now 1790291459719574 us; error 45 us; skew 500 ppm
Sep 24 23:10:59 dist-test-slave-m0z6 krb5kdc[16133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291459, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:59.722329 16146 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:10:59.722867 16146 webserver.cc:460] Webserver started at http://127.12.212.62:44669/ using document root <none> and password file <none>
I20260924 23:10:59.723085 16146 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:59.723134 16146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:59.723232 16146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:59.724311 16146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "59b3808858ed4f2b99dd160875d5b27b"
format_stamp: "Formatted at 2026-09-24 23:10:59 on dist-test-slave-m0z6"
server_key: "265fb2a6b903d2b208deeae75893264d"
server_key_iv: "9c9a663baa776c7ee2e2ba9a155bcc8c"
server_key_version: "encryptionkey@0"
I20260924 23:10:59.724617 16146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "59b3808858ed4f2b99dd160875d5b27b"
format_stamp: "Formatted at 2026-09-24 23:10:59 on dist-test-slave-m0z6"
server_key: "265fb2a6b903d2b208deeae75893264d"
server_key_iv: "9c9a663baa776c7ee2e2ba9a155bcc8c"
server_key_version: "encryptionkey@0"
I20260924 23:10:59.725940 16146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:10:59.726775 16161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:59.726941 16146 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:10:59.727030 16146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "59b3808858ed4f2b99dd160875d5b27b"
format_stamp: "Formatted at 2026-09-24 23:10:59 on dist-test-slave-m0z6"
server_key: "265fb2a6b903d2b208deeae75893264d"
server_key_iv: "9c9a663baa776c7ee2e2ba9a155bcc8c"
server_key_version: "encryptionkey@0"
I20260924 23:10:59.727106 16146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:59.741961 16146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:59.744462 16146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:59.744593 16146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:59.748471 16146 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:46379
I20260924 23:10:59.748502 16213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:46379 every 8 connection(s)
I20260924 23:10:59.748844 16146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:10:59.749837 16214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:10:59.751204 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 16146
I20260924 23:10:59.751286 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:10:59.751482 13136 external_mini_cluster.cc:1428] Setting key 0c75988c9329f89822f4c0cd72b90c67
I20260924 23:10:59.752489 16214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b: Bootstrap starting.
I20260924 23:10:59.753152 16214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b: Neither blocks nor log segments found. Creating new log.
I20260924 23:10:59.753471 16214 log.cc:824] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b: Log is configured to *not* fsync() on all Append() calls
I20260924 23:10:59.754136 16214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b: No bootstrap required, opened a new log
I20260924 23:10:59.756249 16214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59b3808858ed4f2b99dd160875d5b27b" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46379 } }
I20260924 23:10:59.756426 16214 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:10:59.756464 16214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59b3808858ed4f2b99dd160875d5b27b, State: Initialized, Role: FOLLOWER
I20260924 23:10:59.756613 16214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59b3808858ed4f2b99dd160875d5b27b" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46379 } }
I20260924 23:10:59.756695 16214 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:10:59.756726 16214 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:10:59.756794 16214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:10:59.757347 16214 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59b3808858ed4f2b99dd160875d5b27b" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46379 } }
I20260924 23:10:59.757443 16214 leader_election.cc:302] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59b3808858ed4f2b99dd160875d5b27b; no voters:
I20260924 23:10:59.757609 16214 leader_election.cc:288] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:10:59.757632 16219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:10:59.757737 16219 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b [term 1 LEADER]: Becoming Leader. State: Replica: 59b3808858ed4f2b99dd160875d5b27b, State: Running, Role: LEADER
I20260924 23:10:59.757879 16214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:10:59.757870 16219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59b3808858ed4f2b99dd160875d5b27b" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46379 } }
Sep 24 23:10:59 dist-test-slave-m0z6 krb5kdc[16133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:59.758317 16220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59b3808858ed4f2b99dd160875d5b27b. Latest consensus state: current_term: 1 leader_uuid: "59b3808858ed4f2b99dd160875d5b27b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59b3808858ed4f2b99dd160875d5b27b" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46379 } } }
I20260924 23:10:59.758391 16220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b [sys.catalog]: This master's current role is: LEADER
I20260924 23:10:59.758520 16221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59b3808858ed4f2b99dd160875d5b27b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59b3808858ed4f2b99dd160875d5b27b" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46379 } } }
I20260924 23:10:59.758628 16221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b [sys.catalog]: This master's current role is: LEADER
W20260924 23:10:59.759598 16235 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:10:59.759652 16235 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:10:59.759691 16226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:10:59.759819 16226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:10:59.761777 16226 catalog_manager.cc:1347] Generated new cluster ID: 9b87b827da2244009d00833c79bcb9e2
I20260924 23:10:59.761840 16226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:10:59.762864 16217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:59.752036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38936 (local address 127.12.212.62:46379)
0924 23:10:59.752212 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:59.752221 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:59.752247 (+ 26us) server_negotiation.cc:407] Connection header received
0924 23:10:59.752664 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:59.752680 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:59.752731 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:59.752862 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:59.753651 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:59.754602 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:59.755698 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:59.756243 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:59.756340 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:59.758988 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:59.758993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:59.758994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:59.759006 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:59.760943 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:59.761401 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:59.761404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:59.761404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:59.761509 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:59.761810 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:59.761811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:59.761811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:59.762153 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:59.762301 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:59.762475 (+ 174us) server_negotiation.cc:299] Negotiation successful
0924 23:10:59.762621 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260924 23:10:59.771333 16226 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:10:59.771870 16226 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:10:59.776224 16226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59b3808858ed4f2b99dd160875d5b27b: Generated new TSK 0
I20260924 23:10:59.776435 16226 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
I20260924 23:10:59.805792 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:46379
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:44459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:10:59.888360 16240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:10:59.888608 16240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:10:59.888635 16240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:10:59.889889 16240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:10:59.889940 16240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:10:59.889971 16240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:10:59.891647 16240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:44459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:46379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:10:59.891888 16240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:10:59.892092 16240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:10:59.897235 16246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:59.897240 16240 server_base.cc:1034] running on GCE node
W20260924 23:10:59.897222 16245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:10:59.897222 16248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:10:59.897677 16240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:10:59.897921 16240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:10:59.899089 16240 hybrid_clock.cc:648] HybridClock initialized: now 1790291459899050 us; error 53 us; skew 500 ppm
Sep 24 23:10:59 dist-test-slave-m0z6 krb5kdc[16133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291459, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:10:59.901762 16240 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:10:59.902271 16240 webserver.cc:460] Webserver started at http://127.12.212.1:45173/ using document root <none> and password file <none>
I20260924 23:10:59.902463 16240 fs_manager.cc:359] Metadata directory not provided
I20260924 23:10:59.902506 16240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:10:59.902611 16240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:10:59.903745 16240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "3770b33d416349ec956ccfdc8225fb5c"
format_stamp: "Formatted at 2026-09-24 23:10:59 on dist-test-slave-m0z6"
server_key: "984e739244532eac34cc5cf722ca933b"
server_key_iv: "1d3fa631c4101e0e377118d94f3e4c04"
server_key_version: "encryptionkey@0"
I20260924 23:10:59.904072 16240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "3770b33d416349ec956ccfdc8225fb5c"
format_stamp: "Formatted at 2026-09-24 23:10:59 on dist-test-slave-m0z6"
server_key: "984e739244532eac34cc5cf722ca933b"
server_key_iv: "1d3fa631c4101e0e377118d94f3e4c04"
server_key_version: "encryptionkey@0"
I20260924 23:10:59.905366 16240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 23:10:59.906213 16255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:59.906479 16240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:10:59.906564 16240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "3770b33d416349ec956ccfdc8225fb5c"
format_stamp: "Formatted at 2026-09-24 23:10:59 on dist-test-slave-m0z6"
server_key: "984e739244532eac34cc5cf722ca933b"
server_key_iv: "1d3fa631c4101e0e377118d94f3e4c04"
server_key_version: "encryptionkey@0"
I20260924 23:10:59.906642 16240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:10:59.928285 16240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:10:59.930970 16240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:10:59.931116 16240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:10:59.931479 16240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:10:59.931877 16240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:10:59.931927 16240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:59.931959 16240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:10:59.931972 16240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:10:59.937626 16240 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:42027
I20260924 23:10:59.937676 16368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:42027 every 8 connection(s)
I20260924 23:10:59.938023 16240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:10:59.941232 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 16240
I20260924 23:10:59.941340 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:10:59.941526 13136 external_mini_cluster.cc:1428] Setting key b26459b86e7904861ee676dd08e0b911
Sep 24 23:10:59 dist-test-slave-m0z6 krb5kdc[16133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291459, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:10:59.947678 16217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:59.938865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:60037 (local address 127.12.212.62:46379)
0924 23:10:59.938937 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 23:10:59.938939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:10:59.939132 (+ 193us) server_negotiation.cc:407] Connection header received
0924 23:10:59.939646 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:10:59.939647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:10:59.939659 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:10:59.939717 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:10:59.940688 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:59.941200 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:59.942385 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:10:59.942605 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:10:59.942661 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:59.944994 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:10:59.944996 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:10:59.944996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:10:59.945005 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:10:59.946292 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:59.946718 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:59.946720 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:59.946720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:59.946775 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:10:59.947014 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:10:59.947015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:10:59.947016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:10:59.947135 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:10:59.947192 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 23:10:59.947482 (+ 290us) server_negotiation.cc:299] Negotiation successful
0924 23:10:59.947584 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 23:10:59.947867 16371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:10:59.938788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:46379 (local address 127.12.212.1:60037)
0924 23:10:59.939003 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0924 23:10:59.939043 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 23:10:59.939539 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:10:59.939791 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:10:59.939794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:10:59.940372 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:10:59.940582 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:59.940588 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:59.941286 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:59.941287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:59.942308 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:10:59.942311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:10:59.942678 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:10:59.942680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:10:59.942842 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:10:59.943358 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:10:59.943367 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:10:59.944850 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:10:59.946374 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:59.946382 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:59.946385 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:59.946624 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:59.946832 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:10:59.946833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:10:59.946833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:10:59.946957 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:10:59.947192 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:10:59.947199 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:10:59.947347 (+ 148us) client_negotiation.cc:769] Sending connection context
0924 23:10:59.947454 (+ 107us) client_negotiation.cc:240] Negotiation successful
0924 23:10:59.947626 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
I20260924 23:10:59.948360 16369 heartbeater.cc:347] Connected to a master server at 127.12.212.62:46379
I20260924 23:10:59.948483 16369 heartbeater.cc:464] Registering TS with master...
I20260924 23:10:59.948689 16369 heartbeater.cc:511] Master 127.12.212.62:46379 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.12.212.2@KRBTEST.COM; defaulting to no policy
I20260924 23:10:59.949275 16178 ts_manager.cc:194] Registered new tserver with Master: 3770b33d416349ec956ccfdc8225fb5c (127.12.212.1:42027)
I20260924 23:10:59.949898 16178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:60037
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
I20260924 23:10:59.967792 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:46379
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:44459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:00.057951 16374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:00.058189 16374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:00.058215 16374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:00.059486 16374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:00.059543 16374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:00.059572 16374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:11:00.061527 16374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:44459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:46379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:00.061779 16374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:00.061973 16374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:00.067443 16382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:00.067438 16380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:00.067436 16379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:00.067471 16374 server_base.cc:1034] running on GCE node
I20260924 23:11:00.067911 16374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:00.068146 16374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:00.069320 16374 hybrid_clock.cc:648] HybridClock initialized: now 1790291460069300 us; error 33 us; skew 500 ppm
Sep 24 23:11:00 dist-test-slave-m0z6 krb5kdc[16133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291460, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:00.072023 16374 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:11:00.072500 16374 webserver.cc:460] Webserver started at http://127.12.212.2:39263/ using document root <none> and password file <none>
I20260924 23:11:00.072693 16374 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:00.072734 16374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:00.072826 16374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:00.073844 16374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "18a471e363b24249b1dfd13943dcac19"
format_stamp: "Formatted at 2026-09-24 23:11:00 on dist-test-slave-m0z6"
server_key: "768e0dc6d6eb89d2f0c7406e83750aa8"
server_key_iv: "ddae67c5b1a0da6a35ddfdaa6df65586"
server_key_version: "encryptionkey@0"
I20260924 23:11:00.074157 16374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "18a471e363b24249b1dfd13943dcac19"
format_stamp: "Formatted at 2026-09-24 23:11:00 on dist-test-slave-m0z6"
server_key: "768e0dc6d6eb89d2f0c7406e83750aa8"
server_key_iv: "ddae67c5b1a0da6a35ddfdaa6df65586"
server_key_version: "encryptionkey@0"
I20260924 23:11:00.075470 16374 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:11:00.076308 16389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:00.076498 16374 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:11:00.076592 16374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "18a471e363b24249b1dfd13943dcac19"
format_stamp: "Formatted at 2026-09-24 23:11:00 on dist-test-slave-m0z6"
server_key: "768e0dc6d6eb89d2f0c7406e83750aa8"
server_key_iv: "ddae67c5b1a0da6a35ddfdaa6df65586"
server_key_version: "encryptionkey@0"
I20260924 23:11:00.076658 16374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:00.093835 16374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:00.096073 16374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:00.096194 16374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:00.096379 16374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:00.096644 16374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:00.096681 16374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:00.096704 16374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:00.096714 16374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:00.102308 16374 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:44401
I20260924 23:11:00.102347 16502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:44401 every 8 connection(s)
I20260924 23:11:00.102676 16374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:11:00.103734 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 16374
I20260924 23:11:00.103822 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:11:00.104022 13136 external_mini_cluster.cc:1428] Setting key 5ca427ecfcc1a3f8daed6a44a95f2082
Sep 24 23:11:00 dist-test-slave-m0z6 krb5kdc[16133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291460, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.212.3@KRBTEST.COM; defaulting to no policy
I20260924 23:11:00.112150 16217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:00.103422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:36529 (local address 127.12.212.62:46379)
0924 23:11:00.103479 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:00.103480 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:00.103688 (+ 208us) server_negotiation.cc:407] Connection header received
0924 23:11:00.104299 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:00.104301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:00.104311 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:00.104375 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:00.105355 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:00.105948 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:00.107018 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:00.107296 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:00.107366 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:00.109521 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:00.109523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:00.109523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:00.109531 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:00.110889 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:00.111269 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:00.111270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:00.111271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:00.111327 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:00.111619 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:00.111620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:00.111620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:00.111729 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:00.111779 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:00.112011 (+ 232us) server_negotiation.cc:299] Negotiation successful
0924 23:11:00.112081 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 23:11:00.112454 16505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:00.103399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:46379 (local address 127.12.212.2:36529)
0924 23:11:00.103567 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:00.103606 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:00.104199 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:00.104435 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:00.104439 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:00.105057 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:00.105260 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:00.105264 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:00.106047 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:00.106048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:00.106937 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:00.106939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:00.107397 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:00.107398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:00.107515 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:00.108052 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:00.108062 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:00.109425 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:00.110967 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:00.110976 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:00.110978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:00.111203 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:00.111400 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:00.111401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:00.111402 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:00.111555 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:00.111779 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:00.111782 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:00.111912 (+ 130us) client_negotiation.cc:769] Sending connection context
0924 23:11:00.111993 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 23:11:00.112220 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260924 23:11:00.112967 16503 heartbeater.cc:347] Connected to a master server at 127.12.212.62:46379
I20260924 23:11:00.113057 16503 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:00.113224 16503 heartbeater.cc:511] Master 127.12.212.62:46379 requested a full tablet report, sending...
I20260924 23:11:00.113565 16178 ts_manager.cc:194] Registered new tserver with Master: 18a471e363b24249b1dfd13943dcac19 (127.12.212.2:44401)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.3@KRBTEST.COM" created.
I20260924 23:11:00.114058 16178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:36529
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
I20260924 23:11:00.129698 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:46379
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:44459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:00.215231 16508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:00.215451 16508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:00.215476 16508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:00.216652 16508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:00.216689 16508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:00.216714 16508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:11:00.218166 16508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:44459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:46379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:00.218381 16508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:00.218582 16508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:00.223560 16516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:00.223569 16514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:00.223675 16513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:00.223778 16508 server_base.cc:1034] running on GCE node
I20260924 23:11:00.223982 16508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:00.224273 16508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:00.225445 16508 hybrid_clock.cc:648] HybridClock initialized: now 1790291460225428 us; error 29 us; skew 500 ppm
Sep 24 23:11:00 dist-test-slave-m0z6 krb5kdc[16133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291460, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:00.228108 16508 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:11:00.228529 16508 webserver.cc:460] Webserver started at http://127.12.212.3:34933/ using document root <none> and password file <none>
I20260924 23:11:00.228722 16508 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:00.228762 16508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:00.228858 16508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:00.230000 16508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "3d9596f3b3ba41a88cc4b8b0790e0dc6"
format_stamp: "Formatted at 2026-09-24 23:11:00 on dist-test-slave-m0z6"
server_key: "b3a574c2d1c0eb010295f0e5df523c9a"
server_key_iv: "50b8ad1656861c1dbea7eb7984f070f1"
server_key_version: "encryptionkey@0"
I20260924 23:11:00.230358 16508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "3d9596f3b3ba41a88cc4b8b0790e0dc6"
format_stamp: "Formatted at 2026-09-24 23:11:00 on dist-test-slave-m0z6"
server_key: "b3a574c2d1c0eb010295f0e5df523c9a"
server_key_iv: "50b8ad1656861c1dbea7eb7984f070f1"
server_key_version: "encryptionkey@0"
I20260924 23:11:00.231726 16508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:11:00.232455 16523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:00.232602 16508 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:00.232678 16508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "3d9596f3b3ba41a88cc4b8b0790e0dc6"
format_stamp: "Formatted at 2026-09-24 23:11:00 on dist-test-slave-m0z6"
server_key: "b3a574c2d1c0eb010295f0e5df523c9a"
server_key_iv: "50b8ad1656861c1dbea7eb7984f070f1"
server_key_version: "encryptionkey@0"
I20260924 23:11:00.232743 16508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:00.250385 16508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:00.252609 16508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:00.252730 16508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:00.252964 16508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:00.253264 16508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:00.253295 16508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:00.253317 16508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:00.253330 16508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:00.259158 16508 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:34349
I20260924 23:11:00.259243 16636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:34349 every 8 connection(s)
I20260924 23:11:00.259572 16508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:11:00.265265 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 16508
I20260924 23:11:00.265372 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:11:00.265588 13136 external_mini_cluster.cc:1428] Setting key 998f5ee8fbeac12b28bfdacff57816b0
Sep 24 23:11:00 dist-test-slave-m0z6 krb5kdc[16133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291460, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:00.270165 16217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:00.260224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:52963 (local address 127.12.212.62:46379)
0924 23:11:00.260285 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:00.260287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:00.260501 (+ 214us) server_negotiation.cc:407] Connection header received
0924 23:11:00.261027 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:00.261029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:00.261039 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:00.261118 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:00.262150 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:00.262675 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:00.264149 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:00.264354 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:00.264421 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:00.267063 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:00.267064 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:00.267065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:00.267074 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:00.268543 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:00.269052 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:00.269054 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:00.269054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:00.269119 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:00.269414 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:00.269415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:00.269415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:00.269572 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:00.269639 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:00.270015 (+ 376us) server_negotiation.cc:299] Negotiation successful
0924 23:11:00.270095 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 23:11:00.270352 16639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:00.260214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:46379 (local address 127.12.212.3:52963)
0924 23:11:00.260370 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:00.260414 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:00.260937 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:00.261180 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:00.261184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:00.261799 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:00.262012 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:00.262020 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:00.262814 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:00.262815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:00.264040 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:00.264044 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:00.264451 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:00.264452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:00.264580 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:00.265204 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:00.265216 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:00.266922 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:00.268649 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:00.268658 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:00.268660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:00.268921 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:00.269189 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:00.269189 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:00.269190 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:00.269313 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:00.269708 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:00.269715 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:00.269860 (+ 145us) client_negotiation.cc:769] Sending connection context
0924 23:11:00.269992 (+ 132us) client_negotiation.cc:240] Negotiation successful
0924 23:11:00.270133 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
I20260924 23:11:00.270926 16637 heartbeater.cc:347] Connected to a master server at 127.12.212.62:46379
I20260924 23:11:00.271090 16637 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:00.271288 16637 heartbeater.cc:511] Master 127.12.212.62:46379 requested a full tablet report, sending...
I20260924 23:11:00.271682 16178 ts_manager.cc:194] Registered new tserver with Master: 3d9596f3b3ba41a88cc4b8b0790e0dc6 (127.12.212.3:34349)
I20260924 23:11:00.272079 16178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:52963
I20260924 23:11:00.277838 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 23:11:00 dist-test-slave-m0z6 krb5kdc[16133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291460, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 24 23:11:00 dist-test-slave-m0z6 krb5kdc[16133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291460, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:00.300660 16217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:00.293151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38940 (local address 127.12.212.62:46379)
0924 23:11:00.293232 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:00.293234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:00.293308 (+ 74us) server_negotiation.cc:407] Connection header received
0924 23:11:00.293384 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:00.293385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:00.293395 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:00.293443 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:00.294140 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:00.294644 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:00.295420 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:00.295617 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:00.295678 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:00.298183 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:00.298185 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:00.298185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:00.298193 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:00.299501 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:00.299876 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:00.299877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:00.299877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:00.299931 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:00.300104 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:00.300105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:00.300105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:00.300266 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:00.300355 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:00.300474 (+ 119us) server_negotiation.cc:299] Negotiation successful
0924 23:11:00.300567 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260924 23:11:00.300902 16178 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38940
I20260924 23:11:00.301369 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 16240
I20260924 23:11:00.305943 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 16374
I20260924 23:11:00.310199 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 16508
I20260924 23:11:00.314215 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 16146
2026-09-24T23:11:00Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1776 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:11:00 dist-test-slave-m0z6 krb5kdc[16649](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:11:00 dist-test-slave-m0z6 krb5kdc[16649](info): set up 2 sockets
Sep 24 23:11:00 dist-test-slave-m0z6 krb5kdc[16649](info): commencing operation
krb5kdc: starting...
W20260924 23:11:01.270488 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:11:01 dist-test-slave-m0z6 krb5kdc[16649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:11:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:11:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:11:01.392026 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:36939
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:36939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:01.473028 16662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:01.473258 16662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:01.473289 16662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:01.474542 16662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:11:01.474593 16662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:01.474611 16662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:01.474627 16662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:11:01.474641 16662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:11:01.476289 16662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:36939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:36939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:01.476569 16662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:01.476783 16662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:11:01.481674 16662 server_base.cc:1034] running on GCE node
W20260924 23:11:01.481657 16668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:01.481657 16670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:01.481703 16667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:01.482254 16662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:01.482551 16662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:01.483716 16662 hybrid_clock.cc:648] HybridClock initialized: now 1790291461483713 us; error 44 us; skew 500 ppm
Sep 24 23:11:01 dist-test-slave-m0z6 krb5kdc[16649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291461, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:01.490262 16662 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:11:01.490906 16662 webserver.cc:460] Webserver started at http://127.12.212.62:45439/ using document root <none> and password file <none>
I20260924 23:11:01.491145 16662 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:01.491199 16662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:01.491329 16662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:01.492446 16662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "02b53e8a614748c89ef3a85618491bd2"
format_stamp: "Formatted at 2026-09-24 23:11:01 on dist-test-slave-m0z6"
server_key: "5adaa1c79c448fe6d028d29c758e69f7"
server_key_iv: "758c53dae1a18150e50faa6383f6f148"
server_key_version: "encryptionkey@0"
I20260924 23:11:01.492808 16662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "02b53e8a614748c89ef3a85618491bd2"
format_stamp: "Formatted at 2026-09-24 23:11:01 on dist-test-slave-m0z6"
server_key: "5adaa1c79c448fe6d028d29c758e69f7"
server_key_iv: "758c53dae1a18150e50faa6383f6f148"
server_key_version: "encryptionkey@0"
I20260924 23:11:01.494256 16662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:11:01.495118 16677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:01.495328 16662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 23:11:01.495433 16662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "02b53e8a614748c89ef3a85618491bd2"
format_stamp: "Formatted at 2026-09-24 23:11:01 on dist-test-slave-m0z6"
server_key: "5adaa1c79c448fe6d028d29c758e69f7"
server_key_iv: "758c53dae1a18150e50faa6383f6f148"
server_key_version: "encryptionkey@0"
I20260924 23:11:01.495503 16662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:01.504279 16662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:01.506552 16662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:01.506757 16662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:01.510913 16662 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:36939
I20260924 23:11:01.510952 16729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:36939 every 8 connection(s)
I20260924 23:11:01.511475 16662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:11:01.512149 16730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:01.514777 16730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2: Bootstrap starting.
I20260924 23:11:01.515393 16730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:01.515676 16730 log.cc:824] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:01.516508 16730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2: No bootstrap required, opened a new log
I20260924 23:11:01.517010 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 16662
I20260924 23:11:01.517109 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:11:01.517271 13136 external_mini_cluster.cc:1428] Setting key 70f08bedb66ea5ccfa02f8b65fa443dd
I20260924 23:11:01.518889 16730 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02b53e8a614748c89ef3a85618491bd2" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36939 } }
I20260924 23:11:01.519057 16730 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:01.519090 16730 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02b53e8a614748c89ef3a85618491bd2, State: Initialized, Role: FOLLOWER
I20260924 23:11:01.519244 16730 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02b53e8a614748c89ef3a85618491bd2" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36939 } }
I20260924 23:11:01.519332 16730 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:11:01.519367 16730 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:11:01.519420 16730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:01.520020 16730 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02b53e8a614748c89ef3a85618491bd2" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36939 } }
I20260924 23:11:01.520128 16730 leader_election.cc:302] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02b53e8a614748c89ef3a85618491bd2; no voters:
I20260924 23:11:01.520313 16730 leader_election.cc:288] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:11:01.520370 16735 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:01.520565 16730 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:11:01.520612 16735 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2 [term 1 LEADER]: Becoming Leader. State: Replica: 02b53e8a614748c89ef3a85618491bd2, State: Running, Role: LEADER
I20260924 23:11:01.520710 16735 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02b53e8a614748c89ef3a85618491bd2" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36939 } }
I20260924 23:11:01.521090 16735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02b53e8a614748c89ef3a85618491bd2. Latest consensus state: current_term: 1 leader_uuid: "02b53e8a614748c89ef3a85618491bd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02b53e8a614748c89ef3a85618491bd2" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36939 } } }
I20260924 23:11:01.521152 16735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:01.521219 16736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02b53e8a614748c89ef3a85618491bd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02b53e8a614748c89ef3a85618491bd2" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36939 } } }
I20260924 23:11:01.521267 16736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:01.521579 16742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:11:01.522192 16742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:11:01.524194 16742 catalog_manager.cc:1347] Generated new cluster ID: f4726e3868bb421cacb2c808f02a4f60
I20260924 23:11:01.524255 16742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 23:11:01 dist-test-slave-m0z6 krb5kdc[16649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:01.530107 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:01.517786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38058 (local address 127.12.212.62:36939)
0924 23:11:01.517963 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:01.517971 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:01.517998 (+ 27us) server_negotiation.cc:407] Connection header received
0924 23:11:01.518872 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:01.518883 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:01.518948 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:01.519099 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:01.519834 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:01.521359 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:01.522471 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:01.523122 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:01.523247 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:01.526148 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:01.526153 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:01.526154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:01.526169 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:01.528297 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:01.528753 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:01.528757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:01.528757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:01.528913 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:01.529098 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:01.529099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:01.529099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:01.529472 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:01.529640 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:01.529717 (+ 77us) server_negotiation.cc:299] Negotiation successful
0924 23:11:01.529882 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260924 23:11:01.537397 16742 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:11:01.538182 16742 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:11:01.545665 16742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02b53e8a614748c89ef3a85618491bd2: Generated new TSK 0
I20260924 23:11:01.545900 16742 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
I20260924 23:11:01.578639 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:36939
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:01.663600 16756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:01.663838 16756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:01.663867 16756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:01.665159 16756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:01.665211 16756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:01.665243 16756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:11:01.666944 16756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:36939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:01.667200 16756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:01.667435 16756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:01.672521 16761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:01.672559 16764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:01.672667 16756 server_base.cc:1034] running on GCE node
W20260924 23:11:01.672521 16762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:01.673014 16756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:01.673285 16756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:01.674499 16756 hybrid_clock.cc:648] HybridClock initialized: now 1790291461674482 us; error 30 us; skew 500 ppm
Sep 24 23:11:01 dist-test-slave-m0z6 krb5kdc[16649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291461, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:01.677435 16756 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:11:01.677942 16756 webserver.cc:460] Webserver started at http://127.12.212.1:33861/ using document root <none> and password file <none>
I20260924 23:11:01.678206 16756 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:01.678292 16756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:01.678397 16756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:01.679461 16756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "25fac1e6eeb14a6384e061a6f22c853e"
format_stamp: "Formatted at 2026-09-24 23:11:01 on dist-test-slave-m0z6"
server_key: "48d1b48544e706fb7bc78104400200e5"
server_key_iv: "69d4172aaf4c49eec7e2f97bd7ce27f3"
server_key_version: "encryptionkey@0"
I20260924 23:11:01.679764 16756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "25fac1e6eeb14a6384e061a6f22c853e"
format_stamp: "Formatted at 2026-09-24 23:11:01 on dist-test-slave-m0z6"
server_key: "48d1b48544e706fb7bc78104400200e5"
server_key_iv: "69d4172aaf4c49eec7e2f97bd7ce27f3"
server_key_version: "encryptionkey@0"
I20260924 23:11:01.680989 16756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:11:01.681685 16771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:01.681881 16756 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:01.681955 16756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "25fac1e6eeb14a6384e061a6f22c853e"
format_stamp: "Formatted at 2026-09-24 23:11:01 on dist-test-slave-m0z6"
server_key: "48d1b48544e706fb7bc78104400200e5"
server_key_iv: "69d4172aaf4c49eec7e2f97bd7ce27f3"
server_key_version: "encryptionkey@0"
I20260924 23:11:01.682009 16756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:01.705221 16756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:01.707700 16756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:01.707875 16756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:01.708092 16756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:01.708415 16756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:01.708459 16756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:01.708480 16756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:01.708491 16756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:01.713840 16756 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:39705
I20260924 23:11:01.713938 16884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:39705 every 8 connection(s)
I20260924 23:11:01.714229 16756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
Sep 24 23:11:01 dist-test-slave-m0z6 krb5kdc[16649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291461, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:01.724355 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:01.715045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:38349 (local address 127.12.212.62:36939)
0924 23:11:01.715130 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:01.715132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:01.715262 (+ 130us) server_negotiation.cc:407] Connection header received
0924 23:11:01.716232 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:01.716233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:01.716247 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:01.716303 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:01.716922 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:01.717430 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:01.718461 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:01.718660 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:01.718751 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:01.720896 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:01.720898 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:01.720898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:01.720906 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:01.722356 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:01.723013 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:01.723015 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:01.723016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:01.723089 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:01.723464 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:01.723465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:01.723466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:01.723600 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:01.723665 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:01.724160 (+ 495us) server_negotiation.cc:299] Negotiation successful
0924 23:11:01.724262 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260924 23:11:01.724426 16887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:01.714980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36939 (local address 127.12.212.1:38349)
0924 23:11:01.715140 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:01.715181 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:01.716122 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:01.716366 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:01.716369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:01.716642 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:01.716840 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:01.716844 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:01.717533 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:01.717534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:01.718390 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:01.718392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:01.718759 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:01.718760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:01.718870 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:01.719412 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:01.719421 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:01.720774 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:01.722520 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:01.722529 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:01.722531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:01.722821 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:01.723177 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:01.723177 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:01.723178 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:01.723309 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:01.723725 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:01.723732 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:01.723891 (+ 159us) client_negotiation.cc:769] Sending connection context
0924 23:11:01.724042 (+ 151us) client_negotiation.cc:240] Negotiation successful
0924 23:11:01.724209 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":72,"threads_started":1}
I20260924 23:11:01.724476 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 16756
I20260924 23:11:01.724583 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:11:01.724767 13136 external_mini_cluster.cc:1428] Setting key 62fb9eaf6ecd2cd151edab2e6a282acf
I20260924 23:11:01.725270 16885 heartbeater.cc:347] Connected to a master server at 127.12.212.62:36939
I20260924 23:11:01.725370 16885 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:01.725589 16885 heartbeater.cc:511] Master 127.12.212.62:36939 requested a full tablet report, sending...
I20260924 23:11:01.726289 16693 ts_manager.cc:194] Registered new tserver with Master: 25fac1e6eeb14a6384e061a6f22c853e (127.12.212.1:39705)
I20260924 23:11:01.727139 16693 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:38349
WARNING: no policy specified for kudu/127.12.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
I20260924 23:11:01.750355 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:36939
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:01.832338 16890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:01.832577 16890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:01.832604 16890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:01.833868 16890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:01.833925 16890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:01.833954 16890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:11:01.835685 16890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:36939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:01.836104 16890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:01.836390 16890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:01.842479 16898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:01.842479 16895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:01.842860 16890 server_base.cc:1034] running on GCE node
W20260924 23:11:01.842489 16896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:01.843226 16890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:01.843490 16890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:01.844647 16890 hybrid_clock.cc:648] HybridClock initialized: now 1790291461844637 us; error 29 us; skew 500 ppm
Sep 24 23:11:01 dist-test-slave-m0z6 krb5kdc[16649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291461, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:01.847462 16890 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:11:01.847972 16890 webserver.cc:460] Webserver started at http://127.12.212.2:35495/ using document root <none> and password file <none>
I20260924 23:11:01.848189 16890 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:01.848241 16890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:01.848374 16890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:01.849457 16890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "df0783a782fa4093b78e485f7e13275e"
format_stamp: "Formatted at 2026-09-24 23:11:01 on dist-test-slave-m0z6"
server_key: "035cd33e7eee7318e35f675cf72943a4"
server_key_iv: "d2b2d694567da222d7c4036b35071a4a"
server_key_version: "encryptionkey@0"
I20260924 23:11:01.849803 16890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "df0783a782fa4093b78e485f7e13275e"
format_stamp: "Formatted at 2026-09-24 23:11:01 on dist-test-slave-m0z6"
server_key: "035cd33e7eee7318e35f675cf72943a4"
server_key_iv: "d2b2d694567da222d7c4036b35071a4a"
server_key_version: "encryptionkey@0"
I20260924 23:11:01.851243 16890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:11:01.852126 16905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:01.852312 16890 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:11:01.852408 16890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "df0783a782fa4093b78e485f7e13275e"
format_stamp: "Formatted at 2026-09-24 23:11:01 on dist-test-slave-m0z6"
server_key: "035cd33e7eee7318e35f675cf72943a4"
server_key_iv: "d2b2d694567da222d7c4036b35071a4a"
server_key_version: "encryptionkey@0"
I20260924 23:11:01.852486 16890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:01.891583 16890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:01.894158 16890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:01.894349 16890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:01.894596 16890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:01.894948 16890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:01.894984 16890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:01.895015 16890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:01.895031 16890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:01.901446 16890 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:45577
I20260924 23:11:01.901481 17018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:45577 every 8 connection(s)
I20260924 23:11:01.901839 16890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:11:01.906019 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 16890
I20260924 23:11:01.906158 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:11:01.906368 13136 external_mini_cluster.cc:1428] Setting key 2976f91454c45932c9754d76dd03698e
Sep 24 23:11:01 dist-test-slave-m0z6 krb5kdc[16649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291461, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:01.913627 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:01.903224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:40647 (local address 127.12.212.62:36939)
0924 23:11:01.903294 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:01.903296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:01.903491 (+ 195us) server_negotiation.cc:407] Connection header received
0924 23:11:01.904450 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:01.904451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:01.904462 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:01.904523 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:01.905332 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:01.905860 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:01.907385 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:01.907596 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:01.907665 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:01.910324 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:01.910326 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:01.910326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:01.910336 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:01.912103 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:01.912622 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:01.912624 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:01.912625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:01.912687 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:01.912944 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:01.912945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:01.912946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:01.913078 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:01.913137 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:01.913436 (+ 299us) server_negotiation.cc:299] Negotiation successful
0924 23:11:01.913532 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 23:11:01.913813 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:01.903143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36939 (local address 127.12.212.2:40647)
0924 23:11:01.903344 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:01.903387 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:01.904330 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:01.904697 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:01.904701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:01.904999 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:01.905205 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:01.905210 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:01.905998 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:01.905999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:01.907265 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:01.907269 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:01.907691 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:01.907693 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:01.907856 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:01.908457 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:01.908470 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:01.910143 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:01.912214 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:01.912227 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:01.912230 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:01.912540 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:01.912747 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:01.912748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:01.912748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:01.912875 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:01.913139 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:01.913147 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:01.913310 (+ 163us) client_negotiation.cc:769] Sending connection context
0924 23:11:01.913406 (+ 96us) client_negotiation.cc:240] Negotiation successful
0924 23:11:01.913583 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":72,"threads_started":1}
I20260924 23:11:01.914290 17019 heartbeater.cc:347] Connected to a master server at 127.12.212.62:36939
I20260924 23:11:01.914494 17019 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:01.914733 17019 heartbeater.cc:511] Master 127.12.212.62:36939 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.12.212.3@KRBTEST.COM; defaulting to no policy
I20260924 23:11:01.915172 16693 ts_manager.cc:194] Registered new tserver with Master: df0783a782fa4093b78e485f7e13275e (127.12.212.2:45577)
I20260924 23:11:01.915606 16693 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:40647
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
I20260924 23:11:01.939121 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:36939
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:02.024482 17024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:02.024710 17024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:02.024736 17024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:02.025992 17024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:02.026042 17024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:02.026072 17024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:11:02.027832 17024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:36939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:02.028090 17024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:02.028295 17024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:02.033565 17029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:02.033565 17030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:02.033728 17032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:02.034479 17024 server_base.cc:1034] running on GCE node
I20260924 23:11:02.034991 17024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:02.035341 17024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:02.036557 17024 hybrid_clock.cc:648] HybridClock initialized: now 1790291462036534 us; error 51 us; skew 500 ppm
Sep 24 23:11:02 dist-test-slave-m0z6 krb5kdc[16649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291462, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:02.040627 17024 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:11:02.041290 17024 webserver.cc:460] Webserver started at http://127.12.212.3:40563/ using document root <none> and password file <none>
I20260924 23:11:02.041541 17024 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:02.041589 17024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:02.041721 17024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:02.042935 17024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "c3ddedf71ac24c42a60c393f4cb8ee43"
format_stamp: "Formatted at 2026-09-24 23:11:02 on dist-test-slave-m0z6"
server_key: "f513ffd1c9ec65af669b42f636ad0c5c"
server_key_iv: "12923cec5a90062da4fb28a50242e7e7"
server_key_version: "encryptionkey@0"
I20260924 23:11:02.043285 17024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "c3ddedf71ac24c42a60c393f4cb8ee43"
format_stamp: "Formatted at 2026-09-24 23:11:02 on dist-test-slave-m0z6"
server_key: "f513ffd1c9ec65af669b42f636ad0c5c"
server_key_iv: "12923cec5a90062da4fb28a50242e7e7"
server_key_version: "encryptionkey@0"
I20260924 23:11:02.044642 17024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:11:02.045514 17039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:02.045696 17024 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:02.045787 17024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "c3ddedf71ac24c42a60c393f4cb8ee43"
format_stamp: "Formatted at 2026-09-24 23:11:02 on dist-test-slave-m0z6"
server_key: "f513ffd1c9ec65af669b42f636ad0c5c"
server_key_iv: "12923cec5a90062da4fb28a50242e7e7"
server_key_version: "encryptionkey@0"
I20260924 23:11:02.045859 17024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:02.061215 17024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:02.063906 17024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:02.064119 17024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:02.064371 17024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:02.064688 17024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:02.064723 17024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:02.064754 17024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:02.064778 17024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:02.070863 17024 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:41377
I20260924 23:11:02.070907 17152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:41377 every 8 connection(s)
I20260924 23:11:02.071244 17024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:11:02.074448 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 17024
I20260924 23:11:02.074558 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:11:02.074771 13136 external_mini_cluster.cc:1428] Setting key df39d5fbe3c64f854cb168dc1c872676
Sep 24 23:11:02 dist-test-slave-m0z6 krb5kdc[16649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291462, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:02.081338 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:02.072062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:55455 (local address 127.12.212.62:36939)
0924 23:11:02.072107 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:02.072109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:02.072341 (+ 232us) server_negotiation.cc:407] Connection header received
0924 23:11:02.073285 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:02.073286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:02.073298 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:02.073351 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:02.074043 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:02.074544 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:02.075848 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:02.076050 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:02.076103 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:02.078304 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:02.078307 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:02.078307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:02.078317 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:02.079855 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:02.080348 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:02.080350 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:02.080351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:02.080411 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:02.080657 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:02.080658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:02.080659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:02.080788 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:02.080846 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:02.081145 (+ 299us) server_negotiation.cc:299] Negotiation successful
0924 23:11:02.081241 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 23:11:02.081538 17155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:02.072045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36939 (local address 127.12.212.3:55455)
0924 23:11:02.072210 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:02.072249 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:02.073162 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:02.073450 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:02.073454 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:02.073730 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:02.073941 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:02.073945 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:02.074634 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:02.074635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:02.075773 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:02.075775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:02.076119 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:02.076120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:02.076264 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:02.076756 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:02.076764 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:02.078199 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:02.079952 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:02.079962 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:02.079965 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:02.080209 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:02.080468 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:02.080468 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:02.080469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:02.080593 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:02.080846 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:02.080853 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:02.081017 (+ 164us) client_negotiation.cc:769] Sending connection context
0924 23:11:02.081112 (+ 95us) client_negotiation.cc:240] Negotiation successful
0924 23:11:02.081299 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260924 23:11:02.082196 17153 heartbeater.cc:347] Connected to a master server at 127.12.212.62:36939
I20260924 23:11:02.082299 17153 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:02.082499 17153 heartbeater.cc:511] Master 127.12.212.62:36939 requested a full tablet report, sending...
I20260924 23:11:02.083053 16693 ts_manager.cc:194] Registered new tserver with Master: c3ddedf71ac24c42a60c393f4cb8ee43 (127.12.212.3:41377)
I20260924 23:11:02.083406 16693 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:55455
I20260924 23:11:02.086935 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 23:11:02 dist-test-slave-m0z6 krb5kdc[16649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 23:11:02 dist-test-slave-m0z6 krb5kdc[16649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.1@KRBTEST.COM
I20260924 23:11:02.111078 17160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:02.102369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50984 (local address 127.12.212.1:39705)
0924 23:11:02.102526 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:02.102530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:02.102540 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:11:02.102565 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:02.102566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:02.102580 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:02.102647 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:02.103342 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:02.104086 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:02.104828 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:02.105162 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:02.105225 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:02.107803 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:02.107815 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:02.107815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:02.107823 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:02.109654 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:02.110058 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:02.110062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:02.110062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:02.110140 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:02.110441 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:02.110442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:02.110442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:02.110620 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:02.110870 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:02.110916 (+ 46us) server_negotiation.cc:299] Negotiation successful
0924 23:11:02.110992 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
Sep 24 23:11:02 dist-test-slave-m0z6 krb5kdc[16649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:02.119885 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:02.112557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38066 (local address 127.12.212.62:36939)
0924 23:11:02.112615 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:02.112617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:02.112625 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:11:02.112690 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:02.112691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:02.112702 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:02.112746 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:02.113254 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:02.113753 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:02.114462 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:02.114647 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:02.114734 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:02.117061 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:02.117062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:02.117063 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:02.117071 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:02.118325 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:02.118788 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:02.118790 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:02.118790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:02.118863 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:02.119163 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:02.119164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:02.119165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:02.119362 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:02.119455 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:02.119663 (+ 208us) server_negotiation.cc:299] Negotiation successful
0924 23:11:02.119806 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260924 23:11:02.120189 16693 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38066
I20260924 23:11:02.120645 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 16756
I20260924 23:11:02.124845 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 16890
I20260924 23:11:02.128588 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 17024
I20260924 23:11:02.132145 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 16662
2026-09-24T23:11:02Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1818 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:11:02 dist-test-slave-m0z6 krb5kdc[17165](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:11:02 dist-test-slave-m0z6 krb5kdc[17165](info): set up 2 sockets
Sep 24 23:11:02 dist-test-slave-m0z6 krb5kdc[17165](info): commencing operation
krb5kdc: starting...
W20260924 23:11:03.090512 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:11:03 dist-test-slave-m0z6 krb5kdc[17165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:11:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:11:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:11:03.209745 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:41133
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:41133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:03.291491 17178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:03.291826 17178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:03.291855 17178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:03.293197 17178 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:11:03.293260 17178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:03.293274 17178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:03.293286 17178 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:11:03.293298 17178 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:11:03.295308 17178 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:41133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:41133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:03.295580 17178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:03.295783 17178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:03.301447 17183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:03.301494 17178 server_base.cc:1034] running on GCE node
W20260924 23:11:03.301451 17186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:03.301446 17184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:03.301976 17178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:03.302306 17178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:03.303493 17178 hybrid_clock.cc:648] HybridClock initialized: now 1790291463303469 us; error 43 us; skew 500 ppm
Sep 24 23:11:03 dist-test-slave-m0z6 krb5kdc[17165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291463, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:03.306402 17178 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:11:03.306816 17178 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:03.306882 17178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:03.307016 17178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:03.308112 17178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "b4b2e15795cd4bb6a0bc7a2e29290561"
format_stamp: "Formatted at 2026-09-24 23:11:03 on dist-test-slave-m0z6"
server_key: "46b5b2068a3235b3c5ca475a523b1a78"
server_key_iv: "d0769072b7b56161865e39e2304e414f"
server_key_version: "encryptionkey@0"
I20260924 23:11:03.308461 17178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "b4b2e15795cd4bb6a0bc7a2e29290561"
format_stamp: "Formatted at 2026-09-24 23:11:03 on dist-test-slave-m0z6"
server_key: "46b5b2068a3235b3c5ca475a523b1a78"
server_key_iv: "d0769072b7b56161865e39e2304e414f"
server_key_version: "encryptionkey@0"
I20260924 23:11:03.309773 17178 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:11:03.310510 17192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:03.310683 17178 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 23:11:03.310801 17178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "b4b2e15795cd4bb6a0bc7a2e29290561"
format_stamp: "Formatted at 2026-09-24 23:11:03 on dist-test-slave-m0z6"
server_key: "46b5b2068a3235b3c5ca475a523b1a78"
server_key_iv: "d0769072b7b56161865e39e2304e414f"
server_key_version: "encryptionkey@0"
I20260924 23:11:03.310868 17178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:03.339352 17178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:03.341708 17178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:03.341878 17178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:03.345901 17178 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:41133
I20260924 23:11:03.345984 17244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:41133 every 8 connection(s)
I20260924 23:11:03.346267 17178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:11:03.347344 17245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:03.350060 17245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561: Bootstrap starting.
I20260924 23:11:03.350672 17245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:03.351011 17245 log.cc:824] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:03.351752 17245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561: No bootstrap required, opened a new log
I20260924 23:11:03.353372 17245 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4b2e15795cd4bb6a0bc7a2e29290561" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 41133 } }
I20260924 23:11:03.353528 17245 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:03.353550 17245 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4b2e15795cd4bb6a0bc7a2e29290561, State: Initialized, Role: FOLLOWER
I20260924 23:11:03.353703 17245 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4b2e15795cd4bb6a0bc7a2e29290561" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 41133 } }
I20260924 23:11:03.353770 17245 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:11:03.353801 17245 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:11:03.353848 17245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:03.354655 17245 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4b2e15795cd4bb6a0bc7a2e29290561" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 41133 } }
I20260924 23:11:03.354768 17245 leader_election.cc:302] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4b2e15795cd4bb6a0bc7a2e29290561; no voters:
I20260924 23:11:03.354955 17245 leader_election.cc:288] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:11:03.355002 17248 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:03.355109 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 17178
I20260924 23:11:03.355150 17248 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561 [term 1 LEADER]: Becoming Leader. State: Replica: b4b2e15795cd4bb6a0bc7a2e29290561, State: Running, Role: LEADER
I20260924 23:11:03.355216 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:11:03.355221 17245 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:11:03.355237 17248 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4b2e15795cd4bb6a0bc7a2e29290561" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 41133 } }
I20260924 23:11:03.355424 13136 external_mini_cluster.cc:1428] Setting key 6c9f982ca0181f99efe06d7078113052
I20260924 23:11:03.355572 17250 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4b2e15795cd4bb6a0bc7a2e29290561. Latest consensus state: current_term: 1 leader_uuid: "b4b2e15795cd4bb6a0bc7a2e29290561" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4b2e15795cd4bb6a0bc7a2e29290561" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 41133 } } }
I20260924 23:11:03.355661 17250 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:03.355567 17249 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4b2e15795cd4bb6a0bc7a2e29290561" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4b2e15795cd4bb6a0bc7a2e29290561" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 41133 } } }
I20260924 23:11:03.355799 17249 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:03.355995 17255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:11:03.356580 17255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:11:03.358276 17255 catalog_manager.cc:1347] Generated new cluster ID: 2c9939d0ea2e4013b1931efe90eb5932
I20260924 23:11:03.358330 17255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 23:11:03 dist-test-slave-m0z6 krb5kdc[17165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:03.366154 17259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:03.356013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41972 (local address 127.12.212.62:41133)
0924 23:11:03.356521 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:03.356531 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:03.356565 (+ 34us) server_negotiation.cc:407] Connection header received
0924 23:11:03.356764 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:03.356774 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:03.356844 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:03.357061 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:03.357738 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.358892 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.359645 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.360076 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.360173 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:03.362434 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:03.362439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:03.362439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:03.362452 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:03.364588 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:03.364944 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:03.364947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:03.364947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:03.365051 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:03.365232 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:03.365234 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:03.365234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:03.365509 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:03.365675 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:03.365793 (+ 118us) server_negotiation.cc:299] Negotiation successful
0924 23:11:03.365948 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":61,"threads_started":1}
I20260924 23:11:03.374527 17255 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:11:03.375186 17255 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:11:03.386077 17255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561: Generated new TSK 0
I20260924 23:11:03.386329 17255 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
I20260924 23:11:03.415934 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:41133
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:03.498274 17271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:03.498507 17271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:03.498540 17271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:03.499842 17271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:03.499897 17271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:03.499934 17271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:11:03.501616 17271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:41133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:03.501904 17271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:03.502125 17271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:03.507287 17277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:03.507304 17276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:03.507302 17279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:03.507551 17271 server_base.cc:1034] running on GCE node
I20260924 23:11:03.507764 17271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:03.508025 17271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:03.509176 17271 hybrid_clock.cc:648] HybridClock initialized: now 1790291463509182 us; error 49 us; skew 500 ppm
Sep 24 23:11:03 dist-test-slave-m0z6 krb5kdc[17165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291463, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:03.511843 17271 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:11:03.512209 17271 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:03.512259 17271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:03.512362 17271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:03.513401 17271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "a7b56133ebaa46a9a5c1ad25d5851af2"
format_stamp: "Formatted at 2026-09-24 23:11:03 on dist-test-slave-m0z6"
server_key: "8de49db038a5d07627bc248ac5f3f02f"
server_key_iv: "bc689beec1c6573ce75ce631b6cdbdfb"
server_key_version: "encryptionkey@0"
I20260924 23:11:03.513797 17271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "a7b56133ebaa46a9a5c1ad25d5851af2"
format_stamp: "Formatted at 2026-09-24 23:11:03 on dist-test-slave-m0z6"
server_key: "8de49db038a5d07627bc248ac5f3f02f"
server_key_iv: "bc689beec1c6573ce75ce631b6cdbdfb"
server_key_version: "encryptionkey@0"
I20260924 23:11:03.515239 17271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:03.515952 17285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:03.516115 17271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:03.516201 17271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "a7b56133ebaa46a9a5c1ad25d5851af2"
format_stamp: "Formatted at 2026-09-24 23:11:03 on dist-test-slave-m0z6"
server_key: "8de49db038a5d07627bc248ac5f3f02f"
server_key_iv: "bc689beec1c6573ce75ce631b6cdbdfb"
server_key_version: "encryptionkey@0"
I20260924 23:11:03.516256 17271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:03.527385 17271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:03.529667 17271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:03.529790 17271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:03.529991 17271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:03.530292 17271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:03.530325 17271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:03.530349 17271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:03.530371 17271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:03.536183 17271 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:33401
I20260924 23:11:03.536381 17398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:33401 every 8 connection(s)
I20260924 23:11:03.536533 17271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:11:03.541247 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 17271
I20260924 23:11:03.541368 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:11:03.541539 13136 external_mini_cluster.cc:1428] Setting key a7ceb79a128ffa5c0d960ea0efd9da05
Sep 24 23:11:03 dist-test-slave-m0z6 krb5kdc[17165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291463, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:03.547722 17259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:03.537371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:39279 (local address 127.12.212.62:41133)
0924 23:11:03.537433 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:03.537435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:03.537606 (+ 171us) server_negotiation.cc:407] Connection header received
0924 23:11:03.538113 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:03.538115 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:03.538132 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:03.538202 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:03.539546 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.540146 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.541650 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.541923 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.541991 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:03.544638 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:03.544640 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:03.544641 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:03.544652 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:03.546103 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:03.546620 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:03.546621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:03.546621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:03.546673 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:03.547064 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:03.547064 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:03.547065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:03.547184 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:03.547231 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:03.547581 (+ 350us) server_negotiation.cc:299] Negotiation successful
0924 23:11:03.547653 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 23:11:03.548012 17401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:03.537318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:41133 (local address 127.12.212.1:39279)
0924 23:11:03.537477 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:03.537522 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:03.538005 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:03.538469 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:03.538473 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:03.538894 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:03.539425 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:03.539430 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.540256 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.540258 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:03.541540 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:03.541543 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.542032 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.542033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:03.542152 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:03.542687 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:03.542725 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:03.544525 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:03.546188 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:03.546198 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:03.546202 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:03.546544 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:03.546833 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:03.546834 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:03.546834 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:03.546995 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:03.547239 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:03.547246 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:03.547443 (+ 197us) client_negotiation.cc:769] Sending connection context
0924 23:11:03.547572 (+ 129us) client_negotiation.cc:240] Negotiation successful
0924 23:11:03.547758 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260924 23:11:03.548561 17399 heartbeater.cc:347] Connected to a master server at 127.12.212.62:41133
I20260924 23:11:03.548669 17399 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:03.548830 17399 heartbeater.cc:511] Master 127.12.212.62:41133 requested a full tablet report, sending...
I20260924 23:11:03.549422 17209 ts_manager.cc:194] Registered new tserver with Master: a7b56133ebaa46a9a5c1ad25d5851af2 (127.12.212.1:33401)
WARNING: no policy specified for kudu/127.12.212.2@KRBTEST.COM; defaulting to no policy
I20260924 23:11:03.550292 17209 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:39279
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
I20260924 23:11:03.567612 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:41133
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:03.650364 17404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:03.650609 17404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:03.650640 17404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:03.651979 17404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:03.652029 17404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:03.652056 17404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:11:03.653740 17404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:41133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:03.654011 17404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:03.654247 17404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:03.659605 17410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:03.659582 17409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:03.659582 17412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:03.660290 17404 server_base.cc:1034] running on GCE node
I20260924 23:11:03.660527 17404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:03.660799 17404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:03.662042 17404 hybrid_clock.cc:648] HybridClock initialized: now 1790291463662026 us; error 35 us; skew 500 ppm
Sep 24 23:11:03 dist-test-slave-m0z6 krb5kdc[17165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291463, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:03.664883 17404 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:11:03.665246 17404 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:03.665297 17404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:03.665429 17404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:03.666826 17404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "179d737a94cf4bd7a0e3b334100e5741"
format_stamp: "Formatted at 2026-09-24 23:11:03 on dist-test-slave-m0z6"
server_key: "e0f2b06ecaae38c2ef6faab79ee37c2e"
server_key_iv: "22b1642a1aeb86e36cb752a6996893f2"
server_key_version: "encryptionkey@0"
I20260924 23:11:03.667214 17404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "179d737a94cf4bd7a0e3b334100e5741"
format_stamp: "Formatted at 2026-09-24 23:11:03 on dist-test-slave-m0z6"
server_key: "e0f2b06ecaae38c2ef6faab79ee37c2e"
server_key_iv: "22b1642a1aeb86e36cb752a6996893f2"
server_key_version: "encryptionkey@0"
I20260924 23:11:03.668617 17404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 23:11:03.669401 17418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:03.669564 17404 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 23:11:03.669651 17404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "179d737a94cf4bd7a0e3b334100e5741"
format_stamp: "Formatted at 2026-09-24 23:11:03 on dist-test-slave-m0z6"
server_key: "e0f2b06ecaae38c2ef6faab79ee37c2e"
server_key_iv: "22b1642a1aeb86e36cb752a6996893f2"
server_key_version: "encryptionkey@0"
I20260924 23:11:03.669716 17404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:03.688621 17404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:03.690959 17404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:03.691102 17404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:03.691325 17404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:03.691640 17404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:03.691673 17404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:03.691704 17404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:03.691720 17404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:03.697144 17404 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:33265
I20260924 23:11:03.697214 17531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:33265 every 8 connection(s)
I20260924 23:11:03.697489 17404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:11:03.702646 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 17404
I20260924 23:11:03.702764 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:11:03.702936 13136 external_mini_cluster.cc:1428] Setting key cad89a44e08412e8c545809db4c95604
Sep 24 23:11:03 dist-test-slave-m0z6 krb5kdc[17165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291463, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:03.707386 17259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:03.698149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:34573 (local address 127.12.212.62:41133)
0924 23:11:03.698227 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:03.698229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:03.698443 (+ 214us) server_negotiation.cc:407] Connection header received
0924 23:11:03.699131 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:03.699133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:03.699145 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:03.699198 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:03.700195 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.700733 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.701759 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.701951 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.702008 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:03.704458 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:03.704460 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:03.704460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:03.704467 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:03.705977 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:03.706370 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:03.706372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:03.706372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:03.706435 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:03.706662 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:03.706663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:03.706664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:03.706833 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:03.706895 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:03.707197 (+ 302us) server_negotiation.cc:299] Negotiation successful
0924 23:11:03.707286 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 23:11:03.707520 17534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:03.698154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:41133 (local address 127.12.212.2:34573)
0924 23:11:03.698317 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:03.698353 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:03.699013 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:03.699290 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:03.699296 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:03.699573 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:03.700090 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:03.700096 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.700824 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.700825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:03.701669 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:03.701671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.702021 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.702022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:03.702145 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:03.702638 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:03.702647 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:03.704345 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:03.706060 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:03.706066 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:03.706069 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:03.706301 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:03.706494 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:03.706495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:03.706495 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:03.706608 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:03.706900 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:03.706905 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:03.707045 (+ 140us) client_negotiation.cc:769] Sending connection context
0924 23:11:03.707185 (+ 140us) client_negotiation.cc:240] Negotiation successful
0924 23:11:03.707325 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260924 23:11:03.708033 17532 heartbeater.cc:347] Connected to a master server at 127.12.212.62:41133
I20260924 23:11:03.708129 17532 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:03.708294 17532 heartbeater.cc:511] Master 127.12.212.62:41133 requested a full tablet report, sending...
I20260924 23:11:03.708679 17209 ts_manager.cc:194] Registered new tserver with Master: 179d737a94cf4bd7a0e3b334100e5741 (127.12.212.2:33265)
I20260924 23:11:03.709061 17209 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:34573
WARNING: no policy specified for kudu/127.12.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
I20260924 23:11:03.730329 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:41133
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:03.810911 17537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:03.811141 17537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:03.811174 17537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:03.812418 17537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:03.812471 17537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:03.812508 17537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:11:03.814213 17537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:41133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:03.814473 17537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:03.814760 17537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:11:03.820016 17537 server_base.cc:1034] running on GCE node
W20260924 23:11:03.819988 17545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:03.819986 17543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:03.819978 17542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:03.820451 17537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:03.820716 17537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:03.821911 17537 hybrid_clock.cc:648] HybridClock initialized: now 1790291463821894 us; error 28 us; skew 500 ppm
Sep 24 23:11:03 dist-test-slave-m0z6 krb5kdc[17165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291463, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:03.824782 17537 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:11:03.825153 17537 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:03.825205 17537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:03.825325 17537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:03.826538 17537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "f730ef98241544d48be08fcb1b5e3198"
format_stamp: "Formatted at 2026-09-24 23:11:03 on dist-test-slave-m0z6"
server_key: "83729204fec280e13ed6e4241b594407"
server_key_iv: "396f13c0b0764eeb0f70f765a5ec88d0"
server_key_version: "encryptionkey@0"
I20260924 23:11:03.826992 17537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "f730ef98241544d48be08fcb1b5e3198"
format_stamp: "Formatted at 2026-09-24 23:11:03 on dist-test-slave-m0z6"
server_key: "83729204fec280e13ed6e4241b594407"
server_key_iv: "396f13c0b0764eeb0f70f765a5ec88d0"
server_key_version: "encryptionkey@0"
I20260924 23:11:03.828308 17537 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:03.829087 17551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:03.829270 17537 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:03.829355 17537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "f730ef98241544d48be08fcb1b5e3198"
format_stamp: "Formatted at 2026-09-24 23:11:03 on dist-test-slave-m0z6"
server_key: "83729204fec280e13ed6e4241b594407"
server_key_iv: "396f13c0b0764eeb0f70f765a5ec88d0"
server_key_version: "encryptionkey@0"
I20260924 23:11:03.829417 17537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:03.840696 17537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:03.843302 17537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:03.843467 17537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:03.843775 17537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:03.844228 17537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:03.844280 17537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:03.844312 17537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:03.844329 17537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:03.850598 17537 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:37511
I20260924 23:11:03.850673 17664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:37511 every 8 connection(s)
I20260924 23:11:03.851045 17537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:11:03.855316 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 17537
I20260924 23:11:03.855417 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:11:03.855592 13136 external_mini_cluster.cc:1428] Setting key a958b82ed4e8aacb14fcce0e31736e2d
Sep 24 23:11:03 dist-test-slave-m0z6 krb5kdc[17165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291463, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:03.861532 17259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:03.851836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:60929 (local address 127.12.212.62:41133)
0924 23:11:03.851881 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:03.851883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:03.852154 (+ 271us) server_negotiation.cc:407] Connection header received
0924 23:11:03.852671 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:03.852673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:03.852686 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:03.852739 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:03.853782 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.854323 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.855513 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.855809 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.855898 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:03.858295 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:03.858296 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:03.858297 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:03.858305 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:03.859932 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:03.860395 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:03.860397 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:03.860397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:03.860460 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:03.860788 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:03.860789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:03.860790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:03.860926 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:03.860989 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:03.861383 (+ 394us) server_negotiation.cc:299] Negotiation successful
0924 23:11:03.861461 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 23:11:03.861677 17667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:03.851845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:41133 (local address 127.12.212.3:60929)
0924 23:11:03.852020 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:03.852070 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:03.852568 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:03.852814 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:03.852819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:03.853109 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:03.853669 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:03.853677 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.854442 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.854443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:03.855398 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:03.855401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.855910 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.855911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:03.856076 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:03.856603 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:03.856613 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:03.858179 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:03.860030 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:03.860039 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:03.860041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:03.860320 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:03.860526 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:03.860526 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:03.860526 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:03.860642 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:03.861013 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:03.861021 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:03.861203 (+ 182us) client_negotiation.cc:769] Sending connection context
0924 23:11:03.861313 (+ 110us) client_negotiation.cc:240] Negotiation successful
0924 23:11:03.861463 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":78,"threads_started":1}
I20260924 23:11:03.862288 17665 heartbeater.cc:347] Connected to a master server at 127.12.212.62:41133
I20260924 23:11:03.862396 17665 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:03.862561 17665 heartbeater.cc:511] Master 127.12.212.62:41133 requested a full tablet report, sending...
I20260924 23:11:03.862986 17209 ts_manager.cc:194] Registered new tserver with Master: f730ef98241544d48be08fcb1b5e3198 (127.12.212.3:37511)
I20260924 23:11:03.863401 17209 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:60929
I20260924 23:11:03.867453 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 23:11:03.875403 17259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:03.868760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41988 (local address 127.12.212.62:41133)
0924 23:11:03.868850 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:03.868852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:03.868864 (+ 12us) server_negotiation.cc:407] Connection header received
0924 23:11:03.868928 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:03.868929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:03.868938 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:03.868997 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:03.869547 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.870064 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.870964 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.871156 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.871217 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:03.872501 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:03.872503 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:03.872504 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:03.872514 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:03.873946 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:03.874434 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:03.874435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:03.874436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:03.874502 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:03.874823 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:03.874824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:03.874825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:03.874994 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:03.875084 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:03.875264 (+ 180us) server_negotiation.cc:299] Negotiation successful
0924 23:11:03.875334 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260924 23:11:03.877259 17209 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:11:03.878894 17209 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:11:03.885803 17678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:03.881711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53538 (local address 127.12.212.3:37511)
0924 23:11:03.881871 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:03.881876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:03.881923 (+ 47us) server_negotiation.cc:407] Connection header received
0924 23:11:03.881972 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:03.881974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:03.881991 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:03.882085 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:03.882271 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.883162 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.884701 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.885355 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.885418 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:03.885480 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:03.885674 (+ 194us) server_negotiation.cc:299] Negotiation successful
0924 23:11:03.885717 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":71,"threads_started":1}
I20260924 23:11:03.885792 17677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:03.881358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:33265 (local address 127.0.0.1:36952)
0924 23:11:03.881860 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:03.881871 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:03.881929 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:03.882278 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:03.882278 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:03.882280 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:03.882338 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:03.882340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.883500 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.883501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:03.884606 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:03.884609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.885417 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.885418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:03.885512 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:03.885538 (+ 26us) client_negotiation.cc:769] Sending connection context
0924 23:11:03.885624 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 23:11:03.885696 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":4,"thread_start_us":79,"threads_started":1}
I20260924 23:11:03.885793 17676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:03.881555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36952 (local address 127.12.212.2:33265)
0924 23:11:03.882045 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:03.882049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:03.882057 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:11:03.882091 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:03.882093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:03.882108 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:03.882186 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:03.882415 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.883404 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.884714 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.885325 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.885401 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:03.885463 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:03.885659 (+ 196us) server_negotiation.cc:299] Negotiation successful
0924 23:11:03.885703 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":55,"threads_started":1}
I20260924 23:11:03.886155 17674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:03.881576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:37511 (local address 127.0.0.1:53538)
0924 23:11:03.881883 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:03.881889 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:03.881922 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:03.882090 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:03.882093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:03.882095 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:03.882209 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:03.882212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.883237 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.883238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:03.884615 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:03.884617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.885511 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.885512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:03.885577 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:03.885581 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:11:03.885631 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 23:11:03.886106 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":474,"thread_start_us":47,"threads_started":1}
I20260924 23:11:03.886821 17673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:03.881356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:33401 (local address 127.0.0.1:48238)
0924 23:11:03.881687 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:03.881733 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:03.881791 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:03.882996 (+ 1205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:03.882997 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:03.882999 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:03.883044 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:03.883045 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.884041 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.884042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:03.884989 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:03.884992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.886150 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.886151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:03.886469 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:03.886473 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:11:03.886671 (+ 198us) client_negotiation.cc:240] Negotiation successful
0924 23:11:03.886743 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"spinlock_wait_cycles":768,"thread_start_us":79,"threads_started":1}
I20260924 23:11:03.886870 17466 tablet_service.cc:1467] Processing CreateTablet for tablet c10618e2cf1f45fcbf60bccdc80e7eb8 (DEFAULT_TABLE table=test-table [id=c2a0299f6ccd45cbb2cc2453c18a94ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:03.886933 17599 tablet_service.cc:1467] Processing CreateTablet for tablet c10618e2cf1f45fcbf60bccdc80e7eb8 (DEFAULT_TABLE table=test-table [id=c2a0299f6ccd45cbb2cc2453c18a94ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:03.887111 17675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:03.881498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48238 (local address 127.12.212.1:33401)
0924 23:11:03.882829 (+ 1331us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:03.882834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:03.882845 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:11:03.882882 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:03.882884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:03.882899 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:03.882986 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:03.883105 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.883943 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.885085 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:03.885764 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:03.885833 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:03.885889 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:03.886979 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0924 23:11:03.887033 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1289,"thread_start_us":86,"threads_started":1}
I20260924 23:11:03.887255 17466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c10618e2cf1f45fcbf60bccdc80e7eb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:03.887297 17599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c10618e2cf1f45fcbf60bccdc80e7eb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:03.887727 17333 tablet_service.cc:1467] Processing CreateTablet for tablet c10618e2cf1f45fcbf60bccdc80e7eb8 (DEFAULT_TABLE table=test-table [id=c2a0299f6ccd45cbb2cc2453c18a94ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:03.888044 17333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c10618e2cf1f45fcbf60bccdc80e7eb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:03.890086 17679 tablet_bootstrap.cc:492] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198: Bootstrap starting.
I20260924 23:11:03.890602 17680 tablet_bootstrap.cc:492] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741: Bootstrap starting.
I20260924 23:11:03.890750 17679 tablet_bootstrap.cc:654] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:03.891187 17681 tablet_bootstrap.cc:492] T c10618e2cf1f45fcbf60bccdc80e7eb8 P a7b56133ebaa46a9a5c1ad25d5851af2: Bootstrap starting.
I20260924 23:11:03.891343 17679 log.cc:824] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:03.891547 17680 tablet_bootstrap.cc:654] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:03.891793 17681 tablet_bootstrap.cc:654] T c10618e2cf1f45fcbf60bccdc80e7eb8 P a7b56133ebaa46a9a5c1ad25d5851af2: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:03.892162 17680 log.cc:824] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:03.892194 17679 tablet_bootstrap.cc:492] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198: No bootstrap required, opened a new log
I20260924 23:11:03.892264 17679 ts_tablet_manager.cc:1397] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260924 23:11:03.892280 17681 log.cc:824] T c10618e2cf1f45fcbf60bccdc80e7eb8 P a7b56133ebaa46a9a5c1ad25d5851af2: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:03.892935 17680 tablet_bootstrap.cc:492] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741: No bootstrap required, opened a new log
I20260924 23:11:03.893007 17680 ts_tablet_manager.cc:1397] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:11:03.893081 17681 tablet_bootstrap.cc:492] T c10618e2cf1f45fcbf60bccdc80e7eb8 P a7b56133ebaa46a9a5c1ad25d5851af2: No bootstrap required, opened a new log
I20260924 23:11:03.893181 17681 ts_tablet_manager.cc:1397] T c10618e2cf1f45fcbf60bccdc80e7eb8 P a7b56133ebaa46a9a5c1ad25d5851af2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:11:03.893955 17679 raft_consensus.cc:348] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b56133ebaa46a9a5c1ad25d5851af2" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33401 } } peers { permanent_uuid: "179d737a94cf4bd7a0e3b334100e5741" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 33265 } } peers { permanent_uuid: "f730ef98241544d48be08fcb1b5e3198" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37511 } }
I20260924 23:11:03.894094 17679 raft_consensus.cc:374] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:03.894124 17679 raft_consensus.cc:729] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f730ef98241544d48be08fcb1b5e3198, State: Initialized, Role: FOLLOWER
I20260924 23:11:03.894299 17679 consensus_queue.cc:260] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b56133ebaa46a9a5c1ad25d5851af2" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33401 } } peers { permanent_uuid: "179d737a94cf4bd7a0e3b334100e5741" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 33265 } } peers { permanent_uuid: "f730ef98241544d48be08fcb1b5e3198" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37511 } }
I20260924 23:11:03.894567 17679 ts_tablet_manager.cc:1428] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260924 23:11:03.894718 17665 heartbeater.cc:503] Master 127.12.212.62:41133 was elected leader, sending a full tablet report...
I20260924 23:11:03.895023 17680 raft_consensus.cc:348] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b56133ebaa46a9a5c1ad25d5851af2" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33401 } } peers { permanent_uuid: "179d737a94cf4bd7a0e3b334100e5741" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 33265 } } peers { permanent_uuid: "f730ef98241544d48be08fcb1b5e3198" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37511 } }
I20260924 23:11:03.895097 17681 raft_consensus.cc:348] T c10618e2cf1f45fcbf60bccdc80e7eb8 P a7b56133ebaa46a9a5c1ad25d5851af2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b56133ebaa46a9a5c1ad25d5851af2" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33401 } } peers { permanent_uuid: "179d737a94cf4bd7a0e3b334100e5741" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 33265 } } peers { permanent_uuid: "f730ef98241544d48be08fcb1b5e3198" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37511 } }
I20260924 23:11:03.895208 17680 raft_consensus.cc:374] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:03.895231 17681 raft_consensus.cc:374] T c10618e2cf1f45fcbf60bccdc80e7eb8 P a7b56133ebaa46a9a5c1ad25d5851af2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:03.895242 17680 raft_consensus.cc:729] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 179d737a94cf4bd7a0e3b334100e5741, State: Initialized, Role: FOLLOWER
I20260924 23:11:03.895269 17681 raft_consensus.cc:729] T c10618e2cf1f45fcbf60bccdc80e7eb8 P a7b56133ebaa46a9a5c1ad25d5851af2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7b56133ebaa46a9a5c1ad25d5851af2, State: Initialized, Role: FOLLOWER
I20260924 23:11:03.895362 17681 consensus_queue.cc:260] T c10618e2cf1f45fcbf60bccdc80e7eb8 P a7b56133ebaa46a9a5c1ad25d5851af2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b56133ebaa46a9a5c1ad25d5851af2" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33401 } } peers { permanent_uuid: "179d737a94cf4bd7a0e3b334100e5741" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 33265 } } peers { permanent_uuid: "f730ef98241544d48be08fcb1b5e3198" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37511 } }
I20260924 23:11:03.895362 17680 consensus_queue.cc:260] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b56133ebaa46a9a5c1ad25d5851af2" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33401 } } peers { permanent_uuid: "179d737a94cf4bd7a0e3b334100e5741" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 33265 } } peers { permanent_uuid: "f730ef98241544d48be08fcb1b5e3198" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37511 } }
I20260924 23:11:03.895628 17681 ts_tablet_manager.cc:1428] T c10618e2cf1f45fcbf60bccdc80e7eb8 P a7b56133ebaa46a9a5c1ad25d5851af2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:11:03.895640 17680 ts_tablet_manager.cc:1428] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 23:11:03.895748 17399 heartbeater.cc:503] Master 127.12.212.62:41133 was elected leader, sending a full tablet report...
I20260924 23:11:03.895769 17532 heartbeater.cc:503] Master 127.12.212.62:41133 was elected leader, sending a full tablet report...
W20260924 23:11:03.948089 17533 tablet.cc:2369] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 23:11:04.037250 17400 tablet.cc:2369] T c10618e2cf1f45fcbf60bccdc80e7eb8 P a7b56133ebaa46a9a5c1ad25d5851af2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:11:04.073006 17685 raft_consensus.cc:482] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:11:04.073146 17685 raft_consensus.cc:504] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b56133ebaa46a9a5c1ad25d5851af2" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33401 } } peers { permanent_uuid: "179d737a94cf4bd7a0e3b334100e5741" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 33265 } } peers { permanent_uuid: "f730ef98241544d48be08fcb1b5e3198" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37511 } }
I20260924 23:11:04.073544 17685 leader_election.cc:288] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7b56133ebaa46a9a5c1ad25d5851af2 (127.12.212.1:33401), 179d737a94cf4bd7a0e3b334100e5741 (127.12.212.2:33265)
I20260924 23:11:04.076227 17675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.073825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:57573 (local address 127.12.212.1:33401)
0924 23:11:04.073868 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:04.073870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:04.073887 (+ 17us) server_negotiation.cc:407] Connection header received
0924 23:11:04.073913 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:04.073914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:04.073923 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:04.073970 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:04.074097 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:04.074578 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:04.075555 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:04.075915 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:04.075976 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:04.075986 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:04.076124 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 23:11:04.076158 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 23:11:04.076233 17667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.073731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:33401 (local address 127.12.212.3:57573)
0924 23:11:04.073790 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:04.073803 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:04.073849 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:04.073974 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:04.073974 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:04.073976 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:04.074044 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:04.074045 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:04.074653 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:04.074654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:04.075469 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:04.075471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:04.075986 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:04.075988 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:04.076064 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:04.076067 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:11:04.076112 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 23:11:04.076151 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260924 23:11:04.076385 17676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.073840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:54261 (local address 127.12.212.2:33265)
0924 23:11:04.073886 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:04.073887 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:04.073996 (+ 109us) server_negotiation.cc:407] Connection header received
0924 23:11:04.074048 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:04.074049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:04.074057 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:04.074102 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:04.074248 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:04.074792 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:04.075716 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:04.076082 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:04.076141 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:04.076149 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:04.076283 (+ 134us) server_negotiation.cc:299] Negotiation successful
0924 23:11:04.076316 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 23:11:04.076397 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.073748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:33265 (local address 127.12.212.3:54261)
0924 23:11:04.073927 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:04.073936 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:04.073990 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:04.074108 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:04.074109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:04.074110 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:04.074185 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:04.074187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:04.074859 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:04.074860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:04.075635 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:04.075638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:04.076157 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:04.076157 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:04.076225 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:04.076228 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:11:04.076272 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 23:11:04.076310 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":109,"threads_started":1}
I20260924 23:11:04.076586 17353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c10618e2cf1f45fcbf60bccdc80e7eb8" candidate_uuid: "f730ef98241544d48be08fcb1b5e3198" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7b56133ebaa46a9a5c1ad25d5851af2" is_pre_election: true
I20260924 23:11:04.076673 17486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c10618e2cf1f45fcbf60bccdc80e7eb8" candidate_uuid: "f730ef98241544d48be08fcb1b5e3198" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "179d737a94cf4bd7a0e3b334100e5741" is_pre_election: true
I20260924 23:11:04.076748 17353 raft_consensus.cc:2454] T c10618e2cf1f45fcbf60bccdc80e7eb8 P a7b56133ebaa46a9a5c1ad25d5851af2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f730ef98241544d48be08fcb1b5e3198 in term 0.
I20260924 23:11:04.076821 17486 raft_consensus.cc:2454] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f730ef98241544d48be08fcb1b5e3198 in term 0.
I20260924 23:11:04.077001 17552 leader_election.cc:302] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 179d737a94cf4bd7a0e3b334100e5741, f730ef98241544d48be08fcb1b5e3198; no voters:
I20260924 23:11:04.077152 17685 raft_consensus.cc:2781] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:11:04.077194 17685 raft_consensus.cc:482] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:11:04.077216 17685 raft_consensus.cc:3037] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:04.077910 17685 raft_consensus.cc:504] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b56133ebaa46a9a5c1ad25d5851af2" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33401 } } peers { permanent_uuid: "179d737a94cf4bd7a0e3b334100e5741" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 33265 } } peers { permanent_uuid: "f730ef98241544d48be08fcb1b5e3198" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37511 } }
I20260924 23:11:04.078040 17685 leader_election.cc:288] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [CANDIDATE]: Term 1 election: Requested vote from peers a7b56133ebaa46a9a5c1ad25d5851af2 (127.12.212.1:33401), 179d737a94cf4bd7a0e3b334100e5741 (127.12.212.2:33265)
I20260924 23:11:04.078263 17353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c10618e2cf1f45fcbf60bccdc80e7eb8" candidate_uuid: "f730ef98241544d48be08fcb1b5e3198" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7b56133ebaa46a9a5c1ad25d5851af2"
I20260924 23:11:04.078270 17486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c10618e2cf1f45fcbf60bccdc80e7eb8" candidate_uuid: "f730ef98241544d48be08fcb1b5e3198" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "179d737a94cf4bd7a0e3b334100e5741"
I20260924 23:11:04.078339 17486 raft_consensus.cc:3037] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:04.078337 17353 raft_consensus.cc:3037] T c10618e2cf1f45fcbf60bccdc80e7eb8 P a7b56133ebaa46a9a5c1ad25d5851af2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:04.079063 17353 raft_consensus.cc:2454] T c10618e2cf1f45fcbf60bccdc80e7eb8 P a7b56133ebaa46a9a5c1ad25d5851af2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f730ef98241544d48be08fcb1b5e3198 in term 1.
I20260924 23:11:04.079200 17486 raft_consensus.cc:2454] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f730ef98241544d48be08fcb1b5e3198 in term 1.
I20260924 23:11:04.079247 17554 leader_election.cc:302] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7b56133ebaa46a9a5c1ad25d5851af2, f730ef98241544d48be08fcb1b5e3198; no voters:
I20260924 23:11:04.079327 17685 raft_consensus.cc:2781] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:04.079473 17685 raft_consensus.cc:686] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [term 1 LEADER]: Becoming Leader. State: Replica: f730ef98241544d48be08fcb1b5e3198, State: Running, Role: LEADER
I20260924 23:11:04.079576 17685 consensus_queue.cc:237] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b56133ebaa46a9a5c1ad25d5851af2" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33401 } } peers { permanent_uuid: "179d737a94cf4bd7a0e3b334100e5741" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 33265 } } peers { permanent_uuid: "f730ef98241544d48be08fcb1b5e3198" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37511 } }
I20260924 23:11:04.080449 17208 catalog_manager.cc:5495] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 reported cstate change: term changed from 0 to 1, leader changed from <none> to f730ef98241544d48be08fcb1b5e3198 (127.12.212.3). New cstate: current_term: 1 leader_uuid: "f730ef98241544d48be08fcb1b5e3198" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b56133ebaa46a9a5c1ad25d5851af2" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "179d737a94cf4bd7a0e3b334100e5741" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 33265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f730ef98241544d48be08fcb1b5e3198" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 37511 } health_report { overall_health: HEALTHY } } }
W20260924 23:11:04.101658 17666 tablet.cc:2369] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:11:04.113276 17678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.110939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53546 (local address 127.12.212.3:37511)
0924 23:11:04.111005 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:04.111007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:04.111016 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:11:04.111050 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:04.111051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:04.111069 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:04.111130 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:11:04.111291 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:04.111816 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:04.112553 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:04.112743 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:04.112798 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:04.112971 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:11:04.113076 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:11:04.113148 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:04.113169 (+ 21us) server_negotiation.cc:299] Negotiation successful
0924 23:11:04.113190 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 23:11:04.115293 17353 raft_consensus.cc:1261] T c10618e2cf1f45fcbf60bccdc80e7eb8 P a7b56133ebaa46a9a5c1ad25d5851af2 [term 1 FOLLOWER]: Refusing update from remote peer f730ef98241544d48be08fcb1b5e3198: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:04.115303 17486 raft_consensus.cc:1261] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741 [term 1 FOLLOWER]: Refusing update from remote peer f730ef98241544d48be08fcb1b5e3198: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:04.115631 17689 consensus_queue.cc:1035] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7b56133ebaa46a9a5c1ad25d5851af2" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:04.115780 17685 consensus_queue.cc:1035] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [LEADER]: Connected to new peer: Peer: permanent_uuid: "179d737a94cf4bd7a0e3b334100e5741" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 33265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:04.117978 17692 mvcc.cc:205] Tried to move back new op lower bound from 7333033837013422080 to 7333033836870680576. Current Snapshot: MvccSnapshot[applied={T|T < 7333033837013422080}]
I20260924 23:11:04.118171 17694 mvcc.cc:205] Tried to move back new op lower bound from 7333033837013422080 to 7333033836870680576. Current Snapshot: MvccSnapshot[applied={T|T < 7333033837013422080}]
I20260924 23:11:04.120637 17208 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:11:04.120740 17208 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:11:04.121524 17208 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4b2e15795cd4bb6a0bc7a2e29290561: Sending DeleteTablet for 3 replicas of tablet c10618e2cf1f45fcbf60bccdc80e7eb8
I20260924 23:11:04.121860 17466 tablet_service.cc:1514] Processing DeleteTablet for tablet c10618e2cf1f45fcbf60bccdc80e7eb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:04 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:36952
I20260924 23:11:04.121879 17599 tablet_service.cc:1514] Processing DeleteTablet for tablet c10618e2cf1f45fcbf60bccdc80e7eb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:04 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:53538
I20260924 23:11:04.122113 17702 tablet_replica.cc:330] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198: stopping tablet replica
I20260924 23:11:04.122112 17333 tablet_service.cc:1514] Processing DeleteTablet for tablet c10618e2cf1f45fcbf60bccdc80e7eb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:04 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:48238
I20260924 23:11:04.122213 17702 raft_consensus.cc:2229] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:11:04.122336 17702 raft_consensus.cc:2258] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:04.122332 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 17271
I20260924 23:11:04.124595 17701 tablet_replica.cc:330] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741: stopping tablet replica
I20260924 23:11:04.124756 17701 raft_consensus.cc:2229] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:11:04.124888 17701 raft_consensus.cc:2258] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:04.125016 17702 ts_tablet_manager.cc:1905] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:04.125363 17701 ts_tablet_manager.cc:1905] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:04.126510 17702 ts_tablet_manager.cc:1918] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:11:04.126578 17702 log.cc:1196] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/c10618e2cf1f45fcbf60bccdc80e7eb8
I20260924 23:11:04.126927 17702 ts_tablet_manager.cc:1939] T c10618e2cf1f45fcbf60bccdc80e7eb8 P f730ef98241544d48be08fcb1b5e3198: Deleting consensus metadata
I20260924 23:11:04.126991 17701 ts_tablet_manager.cc:1918] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:11:04.127069 17701 log.cc:1196] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/c10618e2cf1f45fcbf60bccdc80e7eb8
I20260924 23:11:04.127337 17701 ts_tablet_manager.cc:1939] T c10618e2cf1f45fcbf60bccdc80e7eb8 P 179d737a94cf4bd7a0e3b334100e5741: Deleting consensus metadata
I20260924 23:11:04.127380 17195 catalog_manager.cc:4841] TS f730ef98241544d48be08fcb1b5e3198 (127.12.212.3:37511): tablet c10618e2cf1f45fcbf60bccdc80e7eb8 (table test-table [id=c2a0299f6ccd45cbb2cc2453c18a94ff]) successfully deleted
I20260924 23:11:04.127875 17193 catalog_manager.cc:4841] TS 179d737a94cf4bd7a0e3b334100e5741 (127.12.212.2:33265): tablet c10618e2cf1f45fcbf60bccdc80e7eb8 (table test-table [id=c2a0299f6ccd45cbb2cc2453c18a94ff]) successfully deleted
W20260924 23:11:04.129920 17195 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.212.1:33401 (error 108)
I20260924 23:11:04.130121 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 17404
I20260924 23:11:04.130456 17673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.130331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:33401 (local address 127.0.0.1:48248)
0924 23:11:04.130383 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:04.130419 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.1:33401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260924 23:11:04.131006 17195 catalog_manager.cc:4584] TS a7b56133ebaa46a9a5c1ad25d5851af2 (127.12.212.1:33401): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c10618e2cf1f45fcbf60bccdc80e7eb8: Network error: Client connection negotiation failed: client connection to 127.12.212.1:33401: connect: Connection refused (error 111)
I20260924 23:11:04.135620 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 17537
I20260924 23:11:04.140738 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 17178
2026-09-24T23:11:04Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2008 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-24T23:11:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:11:04Z Disabled control of system clock
I20260924 23:11:04.167869 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:45431
--webserver_interface=127.12.212.62
--webserver_port=0
--builtin_ntp_servers=127.12.212.20:33531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:45431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 23:11:04.249351 17708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:04.249576 17708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:04.249598 17708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:04.250876 17708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:11:04.250926 17708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:04.250938 17708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:04.250952 17708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:11:04.250962 17708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:11:04.252416 17708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:33531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:45431
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:45431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:04.252668 17708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:04.252969 17708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:04.257643 17713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:04.257771 17714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:04.257795 17708 server_base.cc:1034] running on GCE node
W20260924 23:11:04.257925 17716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:04.258163 17708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:04.258441 17708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:04.259599 17708 hybrid_clock.cc:648] HybridClock initialized: now 1790291464259585 us; error 32 us; skew 500 ppm
I20260924 23:11:04.260887 17708 webserver.cc:460] Webserver started at http://127.12.212.62:43077/ using document root <none> and password file <none>
I20260924 23:11:04.261119 17708 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:04.261168 17708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:04.261294 17708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:04.262363 17708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "d7265a8baa97449fbe6b0c95e3bba03f"
format_stamp: "Formatted at 2026-09-24 23:11:04 on dist-test-slave-m0z6"
server_key: "c51a6a4e49f29dbed1ab9c2daab59b8f"
server_key_iv: "324a2db242f3646cdec1208a6047b484"
server_key_version: "encryptionkey@0"
I20260924 23:11:04.262681 17708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "d7265a8baa97449fbe6b0c95e3bba03f"
format_stamp: "Formatted at 2026-09-24 23:11:04 on dist-test-slave-m0z6"
server_key: "c51a6a4e49f29dbed1ab9c2daab59b8f"
server_key_iv: "324a2db242f3646cdec1208a6047b484"
server_key_version: "encryptionkey@0"
I20260924 23:11:04.263944 17708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:11:04.264799 17722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:04.265012 17708 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:11:04.265106 17708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "d7265a8baa97449fbe6b0c95e3bba03f"
format_stamp: "Formatted at 2026-09-24 23:11:04 on dist-test-slave-m0z6"
server_key: "c51a6a4e49f29dbed1ab9c2daab59b8f"
server_key_iv: "324a2db242f3646cdec1208a6047b484"
server_key_version: "encryptionkey@0"
I20260924 23:11:04.265175 17708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:04.268808 17708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:04.269076 17708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:04.269200 17708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:04.272928 17708 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:45431
I20260924 23:11:04.272957 17774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:45431 every 8 connection(s)
I20260924 23:11:04.273289 17708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:11:04.273977 17775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:04.276880 17775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f: Bootstrap starting.
I20260924 23:11:04.277478 17775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:04.277777 17775 log.cc:824] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:04.278544 17775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f: No bootstrap required, opened a new log
I20260924 23:11:04.280289 17775 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7265a8baa97449fbe6b0c95e3bba03f" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 45431 } }
I20260924 23:11:04.280447 17775 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:04.280470 17775 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7265a8baa97449fbe6b0c95e3bba03f, State: Initialized, Role: FOLLOWER
I20260924 23:11:04.280612 17775 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7265a8baa97449fbe6b0c95e3bba03f" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 45431 } }
I20260924 23:11:04.280683 17775 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:11:04.280705 17775 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:11:04.280735 17775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:04.281333 17775 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7265a8baa97449fbe6b0c95e3bba03f" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 45431 } }
I20260924 23:11:04.281426 17775 leader_election.cc:302] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7265a8baa97449fbe6b0c95e3bba03f; no voters:
I20260924 23:11:04.281594 17775 leader_election.cc:288] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:11:04.281741 17778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:04.281848 17775 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:11:04.281944 17778 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f [term 1 LEADER]: Becoming Leader. State: Replica: d7265a8baa97449fbe6b0c95e3bba03f, State: Running, Role: LEADER
I20260924 23:11:04.282040 17778 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7265a8baa97449fbe6b0c95e3bba03f" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 45431 } }
I20260924 23:11:04.282480 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 17708
I20260924 23:11:04.282574 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:11:04.282766 13136 external_mini_cluster.cc:1428] Setting key ef30406463d8b794fb81b607809fb1a5
I20260924 23:11:04.282991 17779 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7265a8baa97449fbe6b0c95e3bba03f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7265a8baa97449fbe6b0c95e3bba03f" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 45431 } } }
I20260924 23:11:04.283082 17779 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:04.283737 17780 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7265a8baa97449fbe6b0c95e3bba03f. Latest consensus state: current_term: 1 leader_uuid: "d7265a8baa97449fbe6b0c95e3bba03f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7265a8baa97449fbe6b0c95e3bba03f" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 45431 } } }
I20260924 23:11:04.283813 17780 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f [sys.catalog]: This master's current role is: LEADER
W20260924 23:11:04.284416 17796 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:11:04.284469 17796 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:11:04.284569 17786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:11:04.284716 17786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:11:04.285566 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.283391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48758 (local address 127.12.212.62:45431)
0924 23:11:04.283619 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:04.283629 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:04.283795 (+ 166us) server_negotiation.cc:407] Connection header received
0924 23:11:04.283989 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:04.283997 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:04.284049 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:04.284172 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:04.284996 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:04.285000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:04.285001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:04.285005 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:04.285062 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:04.285082 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:04.285153 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:04.285225 (+ 72us) server_negotiation.cc:299] Negotiation successful
0924 23:11:04.285383 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260924 23:11:04.286197 17786 catalog_manager.cc:1347] Generated new cluster ID: ca5e3aeb71484d18b7ffe45bd055607f
I20260924 23:11:04.286242 17786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:11:04.296195 17786 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:11:04.296401 17786 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:11:04.299027 17786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f: Generated new TSK 0
I20260924 23:11:04.299245 17786 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:11:04.305462 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:45431
--builtin_ntp_servers=127.12.212.20:33531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 23:11:04.391328 17799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:04.391530 17799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:04.391554 17799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:04.392818 17799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:04.392859 17799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:04.392886 17799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:11:04.394263 17799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:33531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:45431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:04.394517 17799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:04.394877 17799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:04.399842 17805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:04.399979 17804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:04.399847 17807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:04.400504 17799 server_base.cc:1034] running on GCE node
I20260924 23:11:04.400733 17799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:04.400975 17799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:04.402169 17799 hybrid_clock.cc:648] HybridClock initialized: now 1790291464402148 us; error 37 us; skew 500 ppm
I20260924 23:11:04.403601 17799 webserver.cc:460] Webserver started at http://127.12.212.1:35193/ using document root <none> and password file <none>
I20260924 23:11:04.403862 17799 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:04.403918 17799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:04.404055 17799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:04.405175 17799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "fa852b4e05b34bee901cdb73c1840e6d"
format_stamp: "Formatted at 2026-09-24 23:11:04 on dist-test-slave-m0z6"
server_key: "8da77475f4b62c07b93b4630ce51f4d9"
server_key_iv: "c95d7937127fc97425b64e894503ead1"
server_key_version: "encryptionkey@0"
I20260924 23:11:04.405526 17799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "fa852b4e05b34bee901cdb73c1840e6d"
format_stamp: "Formatted at 2026-09-24 23:11:04 on dist-test-slave-m0z6"
server_key: "8da77475f4b62c07b93b4630ce51f4d9"
server_key_iv: "c95d7937127fc97425b64e894503ead1"
server_key_version: "encryptionkey@0"
I20260924 23:11:04.406930 17799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:11:04.407672 17813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:04.407845 17799 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:04.407939 17799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "fa852b4e05b34bee901cdb73c1840e6d"
format_stamp: "Formatted at 2026-09-24 23:11:04 on dist-test-slave-m0z6"
server_key: "8da77475f4b62c07b93b4630ce51f4d9"
server_key_iv: "c95d7937127fc97425b64e894503ead1"
server_key_version: "encryptionkey@0"
I20260924 23:11:04.408003 17799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:04.411522 17799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:04.411851 17799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:04.411974 17799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:04.412165 17799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:04.412469 17799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:04.412504 17799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:04.412532 17799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:04.412556 17799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:04.418956 17799 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:36995
I20260924 23:11:04.418998 17926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:36995 every 8 connection(s)
I20260924 23:11:04.419451 17799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:11:04.421648 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 17799
I20260924 23:11:04.421749 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:11:04.421944 13136 external_mini_cluster.cc:1428] Setting key a78d5e5fde9c062d93116c1ae47bdef3
I20260924 23:11:04.422756 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.420222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:37805 (local address 127.12.212.62:45431)
0924 23:11:04.420282 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:04.420285 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:04.420522 (+ 237us) server_negotiation.cc:407] Connection header received
0924 23:11:04.421070 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:04.421072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:04.421080 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:04.421144 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:04.422479 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:04.422480 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:04.422481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:04.422485 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:04.422501 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:04.422506 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:04.422553 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:04.422634 (+ 81us) server_negotiation.cc:299] Negotiation successful
0924 23:11:04.422650 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 23:11:04.422847 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:45431
--builtin_ntp_servers=127.12.212.20:33531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260924 23:11:04.423007 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.420213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:45431 (local address 127.12.212.1:37805)
0924 23:11:04.420404 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:04.420440 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:04.420947 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:04.421247 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:04.421252 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:04.421262 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:04.422328 (+ 1066us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:11:04.422328 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:04.422359 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:11:04.422391 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:04.422557 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:04.422562 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:04.422565 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:11:04.422642 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 23:11:04.422739 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260924 23:11:04.423542 17927 heartbeater.cc:347] Connected to a master server at 127.12.212.62:45431
I20260924 23:11:04.423641 17927 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:04.423833 17927 heartbeater.cc:511] Master 127.12.212.62:45431 requested a full tablet report, sending...
I20260924 23:11:04.424396 17739 ts_manager.cc:194] Registered new tserver with Master: fa852b4e05b34bee901cdb73c1840e6d (127.12.212.1:36995)
W20260924 23:11:04.503909 17930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:04.504144 17930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:04.504168 17930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:04.505543 17930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:04.505615 17930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:04.505647 17930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:11:04.507365 17930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:33531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:45431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:04.507665 17930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:04.507997 17930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:04.513470 17935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:04.513480 17936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:04.513468 17938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:04.513520 17930 server_base.cc:1034] running on GCE node
I20260924 23:11:04.513919 17930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:04.514187 17930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:04.515358 17930 hybrid_clock.cc:648] HybridClock initialized: now 1790291464515343 us; error 33 us; skew 500 ppm
I20260924 23:11:04.516767 17930 webserver.cc:460] Webserver started at http://127.12.212.2:37883/ using document root <none> and password file <none>
I20260924 23:11:04.517000 17930 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:04.517053 17930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:04.517184 17930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:04.518301 17930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "669d3e317cb34767bfa67e65b3533a7e"
format_stamp: "Formatted at 2026-09-24 23:11:04 on dist-test-slave-m0z6"
server_key: "5c6c38180d29950fbf695eaffc2dc15f"
server_key_iv: "edd93a36832f05febc86a733e99b3434"
server_key_version: "encryptionkey@0"
I20260924 23:11:04.518661 17930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "669d3e317cb34767bfa67e65b3533a7e"
format_stamp: "Formatted at 2026-09-24 23:11:04 on dist-test-slave-m0z6"
server_key: "5c6c38180d29950fbf695eaffc2dc15f"
server_key_iv: "edd93a36832f05febc86a733e99b3434"
server_key_version: "encryptionkey@0"
I20260924 23:11:04.519995 17930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:04.520793 17944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:04.520977 17930 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:04.521059 17930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "669d3e317cb34767bfa67e65b3533a7e"
format_stamp: "Formatted at 2026-09-24 23:11:04 on dist-test-slave-m0z6"
server_key: "5c6c38180d29950fbf695eaffc2dc15f"
server_key_iv: "edd93a36832f05febc86a733e99b3434"
server_key_version: "encryptionkey@0"
I20260924 23:11:04.521142 17930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:04.524665 17930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:04.524964 17930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:04.525105 17930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:04.525341 17930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:04.525627 17930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:04.525658 17930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:04.525679 17930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:04.525699 17930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:04.531548 17930 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:38599
I20260924 23:11:04.531670 18057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:38599 every 8 connection(s)
I20260924 23:11:04.531910 17930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:11:04.535086 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.532711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:55311 (local address 127.12.212.62:45431)
0924 23:11:04.532764 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:04.532766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:04.533057 (+ 291us) server_negotiation.cc:407] Connection header received
0924 23:11:04.533549 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:04.533551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:04.533559 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:04.533611 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:04.534814 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:04.534816 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:04.534816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:04.534821 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:04.534838 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:04.534843 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:04.534901 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:04.535016 (+ 115us) server_negotiation.cc:299] Negotiation successful
0924 23:11:04.535029 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 23:11:04.535287 18060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.532700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:45431 (local address 127.12.212.2:55311)
0924 23:11:04.532930 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:04.532966 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:04.533445 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:04.533681 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:04.533685 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:04.533694 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:04.534586 (+ 892us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:11:04.534586 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:04.534619 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:11:04.534651 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:04.534932 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:04.534938 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:04.534941 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:11:04.535026 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 23:11:04.535086 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":79,"threads_started":1}
I20260924 23:11:04.535750 18058 heartbeater.cc:347] Connected to a master server at 127.12.212.62:45431
I20260924 23:11:04.535842 18058 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:04.535987 18058 heartbeater.cc:511] Master 127.12.212.62:45431 requested a full tablet report, sending...
I20260924 23:11:04.536334 17739 ts_manager.cc:194] Registered new tserver with Master: 669d3e317cb34767bfa67e65b3533a7e (127.12.212.2:38599)
I20260924 23:11:04.538021 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 17930
I20260924 23:11:04.538110 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:11:04.538272 13136 external_mini_cluster.cc:1428] Setting key 764612322703bf2595437485d607eb75
I20260924 23:11:04.539225 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:45431
--builtin_ntp_servers=127.12.212.20:33531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 23:11:04.624033 18061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:04.624272 18061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:04.624298 18061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:04.625728 18061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:04.625787 18061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:04.625818 18061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:11:04.627384 18061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:33531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:45431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:04.627672 18061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:04.627982 18061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:04.633098 18069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:04.633098 18066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:04.633137 18067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:04.633343 18061 server_base.cc:1034] running on GCE node
I20260924 23:11:04.633579 18061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:04.633843 18061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:04.634989 18061 hybrid_clock.cc:648] HybridClock initialized: now 1790291464634968 us; error 41 us; skew 500 ppm
I20260924 23:11:04.636341 18061 webserver.cc:460] Webserver started at http://127.12.212.3:41397/ using document root <none> and password file <none>
I20260924 23:11:04.636569 18061 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:04.636617 18061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:04.636758 18061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:04.637836 18061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "ecae21657a22401d805845df31c1e997"
format_stamp: "Formatted at 2026-09-24 23:11:04 on dist-test-slave-m0z6"
server_key: "adfca30c254604c6de10df722059450f"
server_key_iv: "b1aa3fbf29243f499b145a95cc18de4d"
server_key_version: "encryptionkey@0"
I20260924 23:11:04.638178 18061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "ecae21657a22401d805845df31c1e997"
format_stamp: "Formatted at 2026-09-24 23:11:04 on dist-test-slave-m0z6"
server_key: "adfca30c254604c6de10df722059450f"
server_key_iv: "b1aa3fbf29243f499b145a95cc18de4d"
server_key_version: "encryptionkey@0"
I20260924 23:11:04.639468 18061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:11:04.640187 18075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:04.640383 18061 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:04.640475 18061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "ecae21657a22401d805845df31c1e997"
format_stamp: "Formatted at 2026-09-24 23:11:04 on dist-test-slave-m0z6"
server_key: "adfca30c254604c6de10df722059450f"
server_key_iv: "b1aa3fbf29243f499b145a95cc18de4d"
server_key_version: "encryptionkey@0"
I20260924 23:11:04.640545 18061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:04.644111 18061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:04.644388 18061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:04.644510 18061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:04.644707 18061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:04.645012 18061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:04.645044 18061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:04.645073 18061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:04.645095 18061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:04.650905 18061 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:35101
I20260924 23:11:04.650943 18188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:35101 every 8 connection(s)
I20260924 23:11:04.651297 18061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:11:04.653914 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 18061
I20260924 23:11:04.654016 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:11:04.654167 13136 external_mini_cluster.cc:1428] Setting key 87d689260f6c2eecf43af5580a736f25
I20260924 23:11:04.654599 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.652134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:41399 (local address 127.12.212.62:45431)
0924 23:11:04.652210 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:04.652213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:04.652469 (+ 256us) server_negotiation.cc:407] Connection header received
0924 23:11:04.652976 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:04.652977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:04.652984 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:04.653052 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:04.654314 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:04.654316 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:04.654316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:04.654322 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:04.654342 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:04.654347 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:04.654380 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:04.654509 (+ 129us) server_negotiation.cc:299] Negotiation successful
0924 23:11:04.654525 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 23:11:04.655038 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.652118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:45431 (local address 127.12.212.3:41399)
0924 23:11:04.652350 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:04.652386 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:04.652864 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:04.653116 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:04.653120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:04.653131 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:04.654089 (+ 958us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:11:04.654090 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:04.654129 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:11:04.654163 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:04.654421 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:04.654428 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:04.654431 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:11:04.654760 (+ 329us) client_negotiation.cc:240] Negotiation successful
0924 23:11:04.654838 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":132,"threads_started":1}
I20260924 23:11:04.655470 18189 heartbeater.cc:347] Connected to a master server at 127.12.212.62:45431
I20260924 23:11:04.655566 18189 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:04.655716 18189 heartbeater.cc:511] Master 127.12.212.62:45431 requested a full tablet report, sending...
I20260924 23:11:04.656096 17739 ts_manager.cc:194] Registered new tserver with Master: ecae21657a22401d805845df31c1e997 (127.12.212.3:35101)
I20260924 23:11:04.666260 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 23:11:04.668757 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.667674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48762 (local address 127.12.212.62:45431)
0924 23:11:04.667783 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:04.667785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:04.667829 (+ 44us) server_negotiation.cc:407] Connection header received
0924 23:11:04.667899 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:04.667900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:04.667906 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:04.667953 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:04.668526 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:04.668528 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:04.668528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:04.668532 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:04.668550 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:04.668554 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:04.668603 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:04.668657 (+ 54us) server_negotiation.cc:299] Negotiation successful
0924 23:11:04.668672 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260924 23:11:04.670409 17739 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:11:04.672065 17739 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:11:04.676055 18201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.674737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:35101 (local address 127.0.0.1:57332)
0924 23:11:04.674858 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:04.674867 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:04.674899 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:04.675250 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:04.675251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:04.675254 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:04.675627 (+ 373us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:11:04.675628 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:04.675635 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:11:04.675655 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:04.675862 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:04.675869 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:04.675872 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:11:04.675932 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 23:11:04.675946 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":81,"threads_started":1}
I20260924 23:11:04.676066 18199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.674533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58010 (local address 127.12.212.2:38599)
0924 23:11:04.674771 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:04.674777 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:04.674790 (+ 13us) server_negotiation.cc:407] Connection header received
0924 23:11:04.674817 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:04.674820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:04.674832 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:04.674911 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:04.675610 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:04.675620 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:04.675620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:04.675625 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:04.675663 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:04.675824 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:04.675903 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:04.675969 (+ 66us) server_negotiation.cc:299] Negotiation successful
0924 23:11:04.675984 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":80,"threads_started":1}
I20260924 23:11:04.676272 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.674425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:38599 (local address 127.0.0.1:58010)
0924 23:11:04.674604 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:04.674660 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:04.674752 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:04.674924 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:04.674930 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:04.674937 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:04.675476 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:11:04.675477 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:04.675495 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:11:04.675522 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:04.675902 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:04.675903 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:04.675904 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 23:11:04.675957 (+ 53us) client_negotiation.cc:240] Negotiation successful
0924 23:11:04.675970 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":64,"threads_started":1}
I20260924 23:11:04.676407 18202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.674943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57332 (local address 127.12.212.3:35101)
0924 23:11:04.675114 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:04.675119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:04.675128 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:11:04.675153 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:04.675155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:04.675163 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:04.675237 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:04.675726 (+ 489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:04.675733 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:04.675734 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:04.675738 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:04.675770 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:04.675800 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:04.675859 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:04.676023 (+ 164us) server_negotiation.cc:299] Negotiation successful
0924 23:11:04.676042 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":75,"threads_started":1}
I20260924 23:11:04.676673 18198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.674433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:36995 (local address 127.0.0.1:42528)
0924 23:11:04.674742 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:04.674752 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:04.674802 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:04.675653 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:04.675654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:04.675657 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:04.676287 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:11:04.676288 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:04.676295 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:11:04.676306 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:04.676550 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:04.676551 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:04.676551 (+ 0us) client_negotiation.cc:769] Sending connection context
0924 23:11:04.676593 (+ 42us) client_negotiation.cc:240] Negotiation successful
0924 23:11:04.676605 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":97,"threads_started":1}
I20260924 23:11:04.676909 18200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.674527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42528 (local address 127.12.212.1:36995)
0924 23:11:04.675283 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:04.675286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:04.675295 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:11:04.675320 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:04.675322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:04.675330 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:04.675400 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:04.676377 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:04.676387 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:04.676388 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:04.676392 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:04.676433 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:04.676466 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:04.676524 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:04.676810 (+ 286us) server_negotiation.cc:299] Negotiation successful
0924 23:11:04.676829 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":93,"threads_started":1}
I20260924 23:11:04.676993 17992 tablet_service.cc:1467] Processing CreateTablet for tablet ea503fb86c444399b5152e4a0b11f0e5 (DEFAULT_TABLE table=test-table [id=b9d1a7a1ec294b97b0b371d7b4d1396f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:04.677028 18123 tablet_service.cc:1467] Processing CreateTablet for tablet ea503fb86c444399b5152e4a0b11f0e5 (DEFAULT_TABLE table=test-table [id=b9d1a7a1ec294b97b0b371d7b4d1396f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:04.677366 17992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea503fb86c444399b5152e4a0b11f0e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:04.677371 18123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea503fb86c444399b5152e4a0b11f0e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:04.677503 17861 tablet_service.cc:1467] Processing CreateTablet for tablet ea503fb86c444399b5152e4a0b11f0e5 (DEFAULT_TABLE table=test-table [id=b9d1a7a1ec294b97b0b371d7b4d1396f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:04.677783 17861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea503fb86c444399b5152e4a0b11f0e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:04.679926 18203 tablet_bootstrap.cc:492] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e: Bootstrap starting.
I20260924 23:11:04.680598 18203 tablet_bootstrap.cc:654] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:04.680598 18204 tablet_bootstrap.cc:492] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997: Bootstrap starting.
I20260924 23:11:04.681073 18205 tablet_bootstrap.cc:492] T ea503fb86c444399b5152e4a0b11f0e5 P fa852b4e05b34bee901cdb73c1840e6d: Bootstrap starting.
I20260924 23:11:04.681233 18204 tablet_bootstrap.cc:654] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:04.681320 18203 log.cc:824] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:04.681531 18204 log.cc:824] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:04.681598 18205 tablet_bootstrap.cc:654] T ea503fb86c444399b5152e4a0b11f0e5 P fa852b4e05b34bee901cdb73c1840e6d: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:04.681931 18205 log.cc:824] T ea503fb86c444399b5152e4a0b11f0e5 P fa852b4e05b34bee901cdb73c1840e6d: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:04.682253 18203 tablet_bootstrap.cc:492] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e: No bootstrap required, opened a new log
I20260924 23:11:04.682331 18203 ts_tablet_manager.cc:1397] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:11:04.682475 18204 tablet_bootstrap.cc:492] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997: No bootstrap required, opened a new log
I20260924 23:11:04.682531 18204 ts_tablet_manager.cc:1397] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:11:04.682627 18205 tablet_bootstrap.cc:492] T ea503fb86c444399b5152e4a0b11f0e5 P fa852b4e05b34bee901cdb73c1840e6d: No bootstrap required, opened a new log
I20260924 23:11:04.682718 18205 ts_tablet_manager.cc:1397] T ea503fb86c444399b5152e4a0b11f0e5 P fa852b4e05b34bee901cdb73c1840e6d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:11:04.684798 18205 raft_consensus.cc:348] T ea503fb86c444399b5152e4a0b11f0e5 P fa852b4e05b34bee901cdb73c1840e6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa852b4e05b34bee901cdb73c1840e6d" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36995 } } peers { permanent_uuid: "669d3e317cb34767bfa67e65b3533a7e" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38599 } } peers { permanent_uuid: "ecae21657a22401d805845df31c1e997" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 35101 } }
I20260924 23:11:04.684933 18205 raft_consensus.cc:374] T ea503fb86c444399b5152e4a0b11f0e5 P fa852b4e05b34bee901cdb73c1840e6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:04.684963 18205 raft_consensus.cc:729] T ea503fb86c444399b5152e4a0b11f0e5 P fa852b4e05b34bee901cdb73c1840e6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa852b4e05b34bee901cdb73c1840e6d, State: Initialized, Role: FOLLOWER
I20260924 23:11:04.685098 18205 consensus_queue.cc:260] T ea503fb86c444399b5152e4a0b11f0e5 P fa852b4e05b34bee901cdb73c1840e6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa852b4e05b34bee901cdb73c1840e6d" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36995 } } peers { permanent_uuid: "669d3e317cb34767bfa67e65b3533a7e" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38599 } } peers { permanent_uuid: "ecae21657a22401d805845df31c1e997" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 35101 } }
I20260924 23:11:04.685052 18204 raft_consensus.cc:348] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa852b4e05b34bee901cdb73c1840e6d" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36995 } } peers { permanent_uuid: "669d3e317cb34767bfa67e65b3533a7e" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38599 } } peers { permanent_uuid: "ecae21657a22401d805845df31c1e997" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 35101 } }
I20260924 23:11:04.685052 18203 raft_consensus.cc:348] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa852b4e05b34bee901cdb73c1840e6d" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36995 } } peers { permanent_uuid: "669d3e317cb34767bfa67e65b3533a7e" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38599 } } peers { permanent_uuid: "ecae21657a22401d805845df31c1e997" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 35101 } }
I20260924 23:11:04.685204 18204 raft_consensus.cc:374] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:04.685204 18203 raft_consensus.cc:374] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:04.685243 18204 raft_consensus.cc:729] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecae21657a22401d805845df31c1e997, State: Initialized, Role: FOLLOWER
I20260924 23:11:04.685243 18203 raft_consensus.cc:729] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 669d3e317cb34767bfa67e65b3533a7e, State: Initialized, Role: FOLLOWER
I20260924 23:11:04.685389 18205 ts_tablet_manager.cc:1428] T ea503fb86c444399b5152e4a0b11f0e5 P fa852b4e05b34bee901cdb73c1840e6d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:11:04.685364 18203 consensus_queue.cc:260] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa852b4e05b34bee901cdb73c1840e6d" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36995 } } peers { permanent_uuid: "669d3e317cb34767bfa67e65b3533a7e" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38599 } } peers { permanent_uuid: "ecae21657a22401d805845df31c1e997" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 35101 } }
I20260924 23:11:04.685364 18204 consensus_queue.cc:260] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa852b4e05b34bee901cdb73c1840e6d" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36995 } } peers { permanent_uuid: "669d3e317cb34767bfa67e65b3533a7e" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38599 } } peers { permanent_uuid: "ecae21657a22401d805845df31c1e997" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 35101 } }
I20260924 23:11:04.685504 17927 heartbeater.cc:503] Master 127.12.212.62:45431 was elected leader, sending a full tablet report...
I20260924 23:11:04.685599 18204 ts_tablet_manager.cc:1428] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:11:04.685599 18203 ts_tablet_manager.cc:1428] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260924 23:11:04.685709 18058 heartbeater.cc:503] Master 127.12.212.62:45431 was elected leader, sending a full tablet report...
I20260924 23:11:04.685730 18189 heartbeater.cc:503] Master 127.12.212.62:45431 was elected leader, sending a full tablet report...
W20260924 23:11:04.782462 18059 tablet.cc:2369] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:11:04.787987 18211 raft_consensus.cc:482] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:11:04.788116 18211 raft_consensus.cc:504] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa852b4e05b34bee901cdb73c1840e6d" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36995 } } peers { permanent_uuid: "669d3e317cb34767bfa67e65b3533a7e" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38599 } } peers { permanent_uuid: "ecae21657a22401d805845df31c1e997" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 35101 } }
I20260924 23:11:04.788504 18211 leader_election.cc:288] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa852b4e05b34bee901cdb73c1840e6d (127.12.212.1:36995), 669d3e317cb34767bfa67e65b3533a7e (127.12.212.2:38599)
I20260924 23:11:04.789783 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.788734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:38599 (local address 127.12.212.3:42777)
0924 23:11:04.788791 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:04.788804 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:04.788864 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:04.788985 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:04.788986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:04.788988 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:04.789425 (+ 437us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:11:04.789425 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:04.789433 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:11:04.789445 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:04.789601 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:04.789601 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:04.789602 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 23:11:04.789703 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 23:11:04.789716 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 23:11:04.789799 18199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.788813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:42777 (local address 127.12.212.2:38599)
0924 23:11:04.788858 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:04.788860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:04.788872 (+ 12us) server_negotiation.cc:407] Connection header received
0924 23:11:04.788921 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:04.788922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:04.788926 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:04.788967 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:04.789520 (+ 553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:04.789520 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:04.789521 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:04.789524 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:04.789541 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:04.789545 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:04.789596 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:04.789702 (+ 106us) server_negotiation.cc:299] Negotiation successful
0924 23:11:04.789717 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 23:11:04.790031 18200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.788868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:42983 (local address 127.12.212.1:36995)
0924 23:11:04.788928 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:04.788931 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:04.789209 (+ 278us) server_negotiation.cc:407] Connection header received
0924 23:11:04.789256 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:04.789257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:04.789263 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:04.789312 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:04.789790 (+ 478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:04.789791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:04.789791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:04.789795 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:04.789814 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:04.789819 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:04.789872 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:04.789957 (+ 85us) server_negotiation.cc:299] Negotiation successful
0924 23:11:04.789971 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 23:11:04.790040 18212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.788741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:36995 (local address 127.12.212.3:42983)
0924 23:11:04.789144 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:04.789153 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:04.789202 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:04.789324 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:04.789325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:04.789327 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:04.789689 (+ 362us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:11:04.789690 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:04.789697 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:11:04.789708 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:04.789898 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:04.789898 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:04.789899 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 23:11:04.789948 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 23:11:04.789960 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":255,"thread_start_us":84,"threads_started":1}
I20260924 23:11:04.790169 18012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea503fb86c444399b5152e4a0b11f0e5" candidate_uuid: "ecae21657a22401d805845df31c1e997" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "669d3e317cb34767bfa67e65b3533a7e" is_pre_election: true
I20260924 23:11:04.790340 18012 raft_consensus.cc:2454] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecae21657a22401d805845df31c1e997 in term 0.
I20260924 23:11:04.790403 17881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea503fb86c444399b5152e4a0b11f0e5" candidate_uuid: "ecae21657a22401d805845df31c1e997" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa852b4e05b34bee901cdb73c1840e6d" is_pre_election: true
I20260924 23:11:04.790555 17881 raft_consensus.cc:2454] T ea503fb86c444399b5152e4a0b11f0e5 P fa852b4e05b34bee901cdb73c1840e6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecae21657a22401d805845df31c1e997 in term 0.
I20260924 23:11:04.790577 18078 leader_election.cc:302] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 669d3e317cb34767bfa67e65b3533a7e, ecae21657a22401d805845df31c1e997; no voters:
I20260924 23:11:04.790735 18211 raft_consensus.cc:2781] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:11:04.790810 18211 raft_consensus.cc:482] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:11:04.790833 18211 raft_consensus.cc:3037] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:04.791556 18211 raft_consensus.cc:504] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa852b4e05b34bee901cdb73c1840e6d" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36995 } } peers { permanent_uuid: "669d3e317cb34767bfa67e65b3533a7e" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38599 } } peers { permanent_uuid: "ecae21657a22401d805845df31c1e997" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 35101 } }
I20260924 23:11:04.791694 18211 leader_election.cc:288] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [CANDIDATE]: Term 1 election: Requested vote from peers fa852b4e05b34bee901cdb73c1840e6d (127.12.212.1:36995), 669d3e317cb34767bfa67e65b3533a7e (127.12.212.2:38599)
I20260924 23:11:04.791844 18012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea503fb86c444399b5152e4a0b11f0e5" candidate_uuid: "ecae21657a22401d805845df31c1e997" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "669d3e317cb34767bfa67e65b3533a7e"
I20260924 23:11:04.791852 17881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea503fb86c444399b5152e4a0b11f0e5" candidate_uuid: "ecae21657a22401d805845df31c1e997" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa852b4e05b34bee901cdb73c1840e6d"
I20260924 23:11:04.791925 17881 raft_consensus.cc:3037] T ea503fb86c444399b5152e4a0b11f0e5 P fa852b4e05b34bee901cdb73c1840e6d [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:04.791929 18012 raft_consensus.cc:3037] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:04.792603 17881 raft_consensus.cc:2454] T ea503fb86c444399b5152e4a0b11f0e5 P fa852b4e05b34bee901cdb73c1840e6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecae21657a22401d805845df31c1e997 in term 1.
I20260924 23:11:04.792696 18012 raft_consensus.cc:2454] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecae21657a22401d805845df31c1e997 in term 1.
I20260924 23:11:04.792801 18076 leader_election.cc:302] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ecae21657a22401d805845df31c1e997, fa852b4e05b34bee901cdb73c1840e6d; no voters:
I20260924 23:11:04.792891 18211 raft_consensus.cc:2781] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:04.793036 18211 raft_consensus.cc:686] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [term 1 LEADER]: Becoming Leader. State: Replica: ecae21657a22401d805845df31c1e997, State: Running, Role: LEADER
I20260924 23:11:04.793135 18211 consensus_queue.cc:237] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa852b4e05b34bee901cdb73c1840e6d" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36995 } } peers { permanent_uuid: "669d3e317cb34767bfa67e65b3533a7e" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38599 } } peers { permanent_uuid: "ecae21657a22401d805845df31c1e997" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 35101 } }
I20260924 23:11:04.793999 17739 catalog_manager.cc:5495] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 reported cstate change: term changed from 0 to 1, leader changed from <none> to ecae21657a22401d805845df31c1e997 (127.12.212.3). New cstate: current_term: 1 leader_uuid: "ecae21657a22401d805845df31c1e997" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa852b4e05b34bee901cdb73c1840e6d" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "669d3e317cb34767bfa67e65b3533a7e" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecae21657a22401d805845df31c1e997" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 35101 } health_report { overall_health: HEALTHY } } }
I20260924 23:11:04.797425 18202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.796502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57336 (local address 127.12.212.3:35101)
0924 23:11:04.796547 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:04.796549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:04.796572 (+ 23us) server_negotiation.cc:407] Connection header received
0924 23:11:04.796603 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:04.796604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:04.796617 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:04.796660 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:04.797224 (+ 564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:04.797226 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:04.797226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:04.797230 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:04.797244 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:04.797248 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:04.797293 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:04.797340 (+ 47us) server_negotiation.cc:299] Negotiation successful
0924 23:11:04.797353 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 23:11:04.799247 17881 raft_consensus.cc:1261] T ea503fb86c444399b5152e4a0b11f0e5 P fa852b4e05b34bee901cdb73c1840e6d [term 1 FOLLOWER]: Refusing update from remote peer ecae21657a22401d805845df31c1e997: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:04.799290 18012 raft_consensus.cc:1261] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e [term 1 FOLLOWER]: Refusing update from remote peer ecae21657a22401d805845df31c1e997: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:04.799595 18213 consensus_queue.cc:1035] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "669d3e317cb34767bfa67e65b3533a7e" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:04.799722 18211 consensus_queue.cc:1035] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa852b4e05b34bee901cdb73c1840e6d" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:04.802060 18216 mvcc.cc:205] Tried to move back new op lower bound from 7333033839814950912 to 7333033839793434624. Current Snapshot: MvccSnapshot[applied={T|T < 7333033839814950912}]
I20260924 23:11:04.802882 18217 mvcc.cc:205] Tried to move back new op lower bound from 7333033839814950912 to 7333033839793434624. Current Snapshot: MvccSnapshot[applied={T|T < 7333033839814950912 or (T in {7333033839814950912})}]
I20260924 23:11:04.804015 18219 mvcc.cc:205] Tried to move back new op lower bound from 7333033839814950912 to 7333033839793434624. Current Snapshot: MvccSnapshot[applied={T|T < 7333033839814950912}]
I20260924 23:11:04.804567 17738 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:11:04.804673 17738 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:11:04.805491 17738 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d7265a8baa97449fbe6b0c95e3bba03f: Sending DeleteTablet for 3 replicas of tablet ea503fb86c444399b5152e4a0b11f0e5
I20260924 23:11:04.805837 18123 tablet_service.cc:1514] Processing DeleteTablet for tablet ea503fb86c444399b5152e4a0b11f0e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:04 UTC) from {username='slave'} at 127.0.0.1:57332
I20260924 23:11:04.805842 17861 tablet_service.cc:1514] Processing DeleteTablet for tablet ea503fb86c444399b5152e4a0b11f0e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:04 UTC) from {username='slave'} at 127.0.0.1:42528
I20260924 23:11:04.805887 17992 tablet_service.cc:1514] Processing DeleteTablet for tablet ea503fb86c444399b5152e4a0b11f0e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:04 UTC) from {username='slave'} at 127.0.0.1:58010
I20260924 23:11:04.806082 18228 tablet_replica.cc:330] T ea503fb86c444399b5152e4a0b11f0e5 P fa852b4e05b34bee901cdb73c1840e6d: stopping tablet replica
I20260924 23:11:04.806195 18228 raft_consensus.cc:2229] T ea503fb86c444399b5152e4a0b11f0e5 P fa852b4e05b34bee901cdb73c1840e6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:11:04.806241 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 17799
I20260924 23:11:04.806301 18228 raft_consensus.cc:2258] T ea503fb86c444399b5152e4a0b11f0e5 P fa852b4e05b34bee901cdb73c1840e6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:04.806458 18227 tablet_replica.cc:330] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e: stopping tablet replica
I20260924 23:11:04.806548 18227 raft_consensus.cc:2229] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:11:04.806631 18227 raft_consensus.cc:2258] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:04.806803 18226 tablet_replica.cc:330] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997: stopping tablet replica
I20260924 23:11:04.806890 18226 raft_consensus.cc:2229] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:11:04.806979 18226 raft_consensus.cc:2258] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:04.807461 18226 ts_tablet_manager.cc:1905] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:04.808168 18227 ts_tablet_manager.cc:1905] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:04.810837 18227 ts_tablet_manager.cc:1918] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:11:04.810987 18227 log.cc:1196] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/ea503fb86c444399b5152e4a0b11f0e5
I20260924 23:11:04.811074 18226 ts_tablet_manager.cc:1918] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:11:04.811154 18226 log.cc:1196] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/ea503fb86c444399b5152e4a0b11f0e5
I20260924 23:11:04.811421 18227 ts_tablet_manager.cc:1939] T ea503fb86c444399b5152e4a0b11f0e5 P 669d3e317cb34767bfa67e65b3533a7e: Deleting consensus metadata
I20260924 23:11:04.811439 18226 ts_tablet_manager.cc:1939] T ea503fb86c444399b5152e4a0b11f0e5 P ecae21657a22401d805845df31c1e997: Deleting consensus metadata
I20260924 23:11:04.811861 17723 catalog_manager.cc:4841] TS ecae21657a22401d805845df31c1e997 (127.12.212.3:35101): tablet ea503fb86c444399b5152e4a0b11f0e5 (table test-table [id=b9d1a7a1ec294b97b0b371d7b4d1396f]) successfully deleted
I20260924 23:11:04.811951 17725 catalog_manager.cc:4841] TS 669d3e317cb34767bfa67e65b3533a7e (127.12.212.2:38599): tablet ea503fb86c444399b5152e4a0b11f0e5 (table test-table [id=b9d1a7a1ec294b97b0b371d7b4d1396f]) successfully deleted
W20260924 23:11:04.812024 17723 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.212.1:36995 (error 108)
I20260924 23:11:04.812247 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 17930
I20260924 23:11:04.812889 18198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.812375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:36995 (local address 127.0.0.1:42534)
0924 23:11:04.812802 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:04.812853 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.1:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404}
W20260924 23:11:04.813735 17723 catalog_manager.cc:4584] TS fa852b4e05b34bee901cdb73c1840e6d (127.12.212.1:36995): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ea503fb86c444399b5152e4a0b11f0e5: Network error: Client connection negotiation failed: client connection to 127.12.212.1:36995: connect: Connection refused (error 111)
I20260924 23:11:04.816545 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 18061
I20260924 23:11:04.821355 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 17708
2026-09-24T23:11:04Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (679 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260924 23:11:04.840025 13136 mini_oidc.cc:152] Starting JWKS server
I20260924 23:11:04.841153 13136 webserver.cc:460] Webserver started at https://127.0.0.1:35227/ using document root <none> and password file <none>
I20260924 23:11:04.841260 13136 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 23:11:04.841434 13136 webserver.cc:460] Webserver started at http://127.0.0.1:33917/ using document root <none> and password file <none>
2026-09-24T23:11:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:11:04Z Disabled control of system clock
I20260924 23:11:04.848793 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:33483
--webserver_interface=127.12.212.62
--webserver_port=0
--builtin_ntp_servers=127.12.212.20:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:33483
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35227/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0/testchainca.cert with env {}
W20260924 23:11:04.931461 18235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:04.931703 18235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:04.931737 18235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:04.933030 18235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:11:04.933084 18235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:04.933095 18235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:04.933106 18235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:11:04.933117 18235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 23:11:04.933130 18235 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 23:11:04.933138 18235 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35227/jwks.json
I20260924 23:11:04.934921 18235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:33483
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:33483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35227/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:04.935184 18235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:04.935401 18235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:04.940274 18240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:04.940347 18241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:04.940429 18235 server_base.cc:1034] running on GCE node
W20260924 23:11:04.940534 18243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:04.940788 18235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:04.941108 18235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:04.942265 18235 hybrid_clock.cc:648] HybridClock initialized: now 1790291464942248 us; error 31 us; skew 500 ppm
I20260924 23:11:04.943539 18235 webserver.cc:460] Webserver started at http://127.12.212.62:35037/ using document root <none> and password file <none>
I20260924 23:11:04.943784 18235 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:04.943831 18235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:04.943931 18235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:04.944994 18235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "3c4a9f86d5c045fc9794595b65d9de6a"
format_stamp: "Formatted at 2026-09-24 23:11:04 on dist-test-slave-m0z6"
server_key: "57d105bd9f442d53077a18c0328c6423"
server_key_iv: "caeafaf937653ee8a35d9db8cbc8a5fb"
server_key_version: "encryptionkey@0"
I20260924 23:11:04.945302 18235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "3c4a9f86d5c045fc9794595b65d9de6a"
format_stamp: "Formatted at 2026-09-24 23:11:04 on dist-test-slave-m0z6"
server_key: "57d105bd9f442d53077a18c0328c6423"
server_key_iv: "caeafaf937653ee8a35d9db8cbc8a5fb"
server_key_version: "encryptionkey@0"
I20260924 23:11:04.946511 18235 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:11:04.947338 18249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:04.947491 18235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:04.947572 18235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "3c4a9f86d5c045fc9794595b65d9de6a"
format_stamp: "Formatted at 2026-09-24 23:11:04 on dist-test-slave-m0z6"
server_key: "57d105bd9f442d53077a18c0328c6423"
server_key_iv: "caeafaf937653ee8a35d9db8cbc8a5fb"
server_key_version: "encryptionkey@0"
I20260924 23:11:04.947649 18235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:04.962136 18235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:04.962468 18235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:04.962603 18235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:04.966333 18235 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:33483
I20260924 23:11:04.966403 18301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:33483 every 8 connection(s)
I20260924 23:11:04.966749 18235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:11:04.967764 18302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:04.970108 18302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a: Bootstrap starting.
I20260924 23:11:04.970678 18302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:04.970976 18302 log.cc:824] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:04.971901 18302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a: No bootstrap required, opened a new log
I20260924 23:11:04.973146 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 18235
I20260924 23:11:04.973255 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:11:04.973450 13136 external_mini_cluster.cc:1428] Setting key 7dfb2f97b56e07792d5032ea18a64e09
I20260924 23:11:04.973981 18302 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a9f86d5c045fc9794595b65d9de6a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 33483 } }
I20260924 23:11:04.974172 18302 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:04.974216 18302 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c4a9f86d5c045fc9794595b65d9de6a, State: Initialized, Role: FOLLOWER
I20260924 23:11:04.974344 18302 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a9f86d5c045fc9794595b65d9de6a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 33483 } }
I20260924 23:11:04.974424 18302 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:11:04.974459 18302 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:11:04.974495 18302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:04.975498 18302 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a9f86d5c045fc9794595b65d9de6a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 33483 } }
I20260924 23:11:04.975605 18302 leader_election.cc:302] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c4a9f86d5c045fc9794595b65d9de6a; no voters:
I20260924 23:11:04.975772 18302 leader_election.cc:288] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:11:04.975834 18307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:04.975967 18307 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a [term 1 LEADER]: Becoming Leader. State: Replica: 3c4a9f86d5c045fc9794595b65d9de6a, State: Running, Role: LEADER
I20260924 23:11:04.976018 18302 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:11:04.976063 18307 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a9f86d5c045fc9794595b65d9de6a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 33483 } }
I20260924 23:11:04.976576 18308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c4a9f86d5c045fc9794595b65d9de6a. Latest consensus state: current_term: 1 leader_uuid: "3c4a9f86d5c045fc9794595b65d9de6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a9f86d5c045fc9794595b65d9de6a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 33483 } } }
I20260924 23:11:04.976753 18309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c4a9f86d5c045fc9794595b65d9de6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a9f86d5c045fc9794595b65d9de6a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 33483 } } }
I20260924 23:11:04.976866 18309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:04.976818 18308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a [sys.catalog]: This master's current role is: LEADER
W20260924 23:11:04.977840 18322 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:11:04.977891 18322 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:11:04.978775 18317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:11:04.978986 18317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:11:04.980737 18306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:04.974040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48982 (local address 127.12.212.62:33483)
0924 23:11:04.974376 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:04.974385 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:04.974419 (+ 34us) server_negotiation.cc:407] Connection header received
0924 23:11:04.974553 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:04.974560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:04.974639 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:04.975061 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:04.975174 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:04.976908 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:04.978058 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:04.978686 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:04.978832 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:04.979730 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:04.979734 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:04.979735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:04.979942 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:04.979988 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:04.980005 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:04.980066 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:04.980420 (+ 354us) server_negotiation.cc:299] Negotiation successful
0924 23:11:04.980516 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":96,"threads_started":1}
I20260924 23:11:04.981745 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:11:04.982534 18317 catalog_manager.cc:1347] Generated new cluster ID: 81ac1f939a9246a88976978cc3b3fa77
I20260924 23:11:04.982585 18317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:11:04.994941 18317 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:11:04.995530 18317 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:11:05.001688 18317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c4a9f86d5c045fc9794595b65d9de6a: Generated new TSK 0
I20260924 23:11:05.001916 18317 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260924 23:11:05.024241 18332 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260924 23:11:05.024346 18332 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260924 23:11:05.025029 18331 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:05.008025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:33483 (local address 127.0.0.1:48984)
0924 23:11:05.008213 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:05.008225 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:05.008288 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:05.008556 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:05.008557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:05.008565 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0924 23:11:05.008627 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:05.008629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.009742 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.009743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:05.010430 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:05.010433 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.010943 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.010945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:05.011005 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:05.011036 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 23:11:05.024549 (+ 13513us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35227/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0924 23:11:05.024679 (+ 130us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.212.62:33483: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35227/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.12.212.62:33483: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35227/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260924 23:11:05.025493 18306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:05.008210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48984 (local address 127.12.212.62:33483)
0924 23:11:05.008271 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:05.008274 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:05.008287 (+ 13us) server_negotiation.cc:407] Connection header received
0924 23:11:05.008408 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:05.008410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:05.008423 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:05.008513 (+ 90us) server_negotiation.cc:226] Negotiated authn=JWT
0924 23:11:05.008764 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.009622 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.010521 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.010846 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.010916 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:05.011167 (+ 251us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 23:11:05.024377 (+ 13210us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35227/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0924 23:11:05.025307 (+ 930us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48984: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35227/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 23:11:05.025514 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 18235
W20260924 23:11:05.025550 18306 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48984: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35227/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
2026-09-24T23:11:05Z chronyd exiting
I20260924 23:11:05.061162 13136 test_util.cc:175] -----------------------------------------------
I20260924 23:11:05.061254 13136 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790291444018253-13136-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (222 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260924 23:11:05.062021 13136 mini_oidc.cc:152] Starting JWKS server
I20260924 23:11:05.062769 13136 webserver.cc:460] Webserver started at https://127.0.0.1:44309/ using document root <none> and password file <none>
I20260924 23:11:05.062825 13136 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 23:11:05.062974 13136 webserver.cc:460] Webserver started at http://127.0.0.1:44449/ using document root <none> and password file <none>
2026-09-24T23:11:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:11:05Z Disabled control of system clock
I20260924 23:11:05.071033 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:44171
--webserver_interface=127.12.212.62
--webserver_port=0
--builtin_ntp_servers=127.12.212.20:36553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:44171
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44309/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790291444018253-13136-0/testchainca.cert with env {}
W20260924 23:11:05.150890 18338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:05.151103 18338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:05.151125 18338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:05.152359 18338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:11:05.152410 18338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:05.152421 18338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:05.152432 18338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:11:05.152446 18338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 23:11:05.152468 18338 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 23:11:05.152491 18338 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44309/jwks.json
I20260924 23:11:05.154062 18338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:36553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:44171
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:44171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44309/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790291444018253-13136-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:05.154325 18338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:05.154526 18338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:05.159353 18344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:05.159361 18343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:05.159468 18338 server_base.cc:1034] running on GCE node
W20260924 23:11:05.159363 18346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:05.159893 18338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:05.160216 18338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:05.161397 18338 hybrid_clock.cc:648] HybridClock initialized: now 1790291465161385 us; error 27 us; skew 500 ppm
I20260924 23:11:05.162730 18338 webserver.cc:460] Webserver started at http://127.12.212.62:33719/ using document root <none> and password file <none>
I20260924 23:11:05.162967 18338 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:05.163013 18338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:05.163216 18338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:05.164354 18338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "8422c7949db44597beea68f2260acf57"
format_stamp: "Formatted at 2026-09-24 23:11:05 on dist-test-slave-m0z6"
server_key: "98431ca3f9ff9236c6f222162f77a8bc"
server_key_iv: "8c6ec65727de498cf4afde03c0f7992d"
server_key_version: "encryptionkey@0"
I20260924 23:11:05.164705 18338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "8422c7949db44597beea68f2260acf57"
format_stamp: "Formatted at 2026-09-24 23:11:05 on dist-test-slave-m0z6"
server_key: "98431ca3f9ff9236c6f222162f77a8bc"
server_key_iv: "8c6ec65727de498cf4afde03c0f7992d"
server_key_version: "encryptionkey@0"
I20260924 23:11:05.165978 18338 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:11:05.166721 18352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:05.166862 18338 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:05.166939 18338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "8422c7949db44597beea68f2260acf57"
format_stamp: "Formatted at 2026-09-24 23:11:05 on dist-test-slave-m0z6"
server_key: "98431ca3f9ff9236c6f222162f77a8bc"
server_key_iv: "8c6ec65727de498cf4afde03c0f7992d"
server_key_version: "encryptionkey@0"
I20260924 23:11:05.167009 18338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:05.195426 18338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:05.195755 18338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:05.195876 18338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:05.199720 18338 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:44171
I20260924 23:11:05.199771 18404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:44171 every 8 connection(s)
I20260924 23:11:05.200110 18338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:11:05.200799 18405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:05.203599 18405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57: Bootstrap starting.
I20260924 23:11:05.204131 18405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:05.204361 18405 log.cc:824] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:05.205041 18405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57: No bootstrap required, opened a new log
I20260924 23:11:05.205626 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 18338
I20260924 23:11:05.205725 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:11:05.205878 13136 external_mini_cluster.cc:1428] Setting key b2693689d3d5b81cecd8083c055d8296
I20260924 23:11:05.207108 18405 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8422c7949db44597beea68f2260acf57" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 44171 } }
I20260924 23:11:05.207296 18405 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:05.207336 18405 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8422c7949db44597beea68f2260acf57, State: Initialized, Role: FOLLOWER
I20260924 23:11:05.207484 18405 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8422c7949db44597beea68f2260acf57" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 44171 } }
I20260924 23:11:05.207572 18405 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:11:05.207607 18405 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:11:05.207660 18405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:05.208412 18405 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8422c7949db44597beea68f2260acf57" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 44171 } }
I20260924 23:11:05.208529 18405 leader_election.cc:302] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8422c7949db44597beea68f2260acf57; no voters:
I20260924 23:11:05.208698 18405 leader_election.cc:288] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:11:05.208760 18410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:05.208973 18405 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:11:05.209043 18410 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57 [term 1 LEADER]: Becoming Leader. State: Replica: 8422c7949db44597beea68f2260acf57, State: Running, Role: LEADER
I20260924 23:11:05.209230 18410 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8422c7949db44597beea68f2260acf57" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 44171 } }
I20260924 23:11:05.209724 18411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8422c7949db44597beea68f2260acf57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8422c7949db44597beea68f2260acf57" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 44171 } } }
I20260924 23:11:05.209753 18412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8422c7949db44597beea68f2260acf57. Latest consensus state: current_term: 1 leader_uuid: "8422c7949db44597beea68f2260acf57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8422c7949db44597beea68f2260acf57" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 44171 } } }
I20260924 23:11:05.209827 18411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:05.209837 18412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:05.210358 18420 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:11:05.210896 18420 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:11:05.212453 18420 catalog_manager.cc:1347] Generated new cluster ID: 00c7be2ad023409085589b73b62085d7
I20260924 23:11:05.212504 18420 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:11:05.212980 18409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:05.206362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59044 (local address 127.12.212.62:44171)
0924 23:11:05.206602 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:05.206613 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:05.206647 (+ 34us) server_negotiation.cc:407] Connection header received
0924 23:11:05.207072 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:05.207081 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:05.207196 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:05.207356 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:05.207610 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.209497 (+ 1887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.211035 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.211478 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.211598 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:05.212262 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:05.212265 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:05.212266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:05.212462 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:05.212504 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:05.212520 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:05.212589 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:05.212692 (+ 103us) server_negotiation.cc:299] Negotiation successful
0924 23:11:05.212776 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260924 23:11:05.213817 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:11:05.220883 18420 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:11:05.221450 18420 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:11:05.224171 18420 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8422c7949db44597beea68f2260acf57: Generated new TSK 0
I20260924 23:11:05.224355 18420 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260924 23:11:05.252907 18435 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260924 23:11:05.252996 18435 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260924 23:11:05.253284 18409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:05.231425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59048 (local address 127.12.212.62:44171)
0924 23:11:05.231479 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:05.231481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:05.231588 (+ 107us) server_negotiation.cc:407] Connection header received
0924 23:11:05.231667 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:05.231667 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:05.231678 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:05.231740 (+ 62us) server_negotiation.cc:226] Negotiated authn=JWT
0924 23:11:05.231925 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.232397 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.233200 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.233389 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.233445 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:05.233609 (+ 164us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 23:11:05.253067 (+ 19458us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0924 23:11:05.253226 (+ 159us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59048: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":26}
W20260924 23:11:05.253334 18409 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59048: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260924 23:11:05.253413 18434 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:05.231361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:44171 (local address 127.0.0.1:59048)
0924 23:11:05.231519 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:05.231530 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:05.231599 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:05.231796 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:05.231797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:05.231798 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0924 23:11:05.231859 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:05.231861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.232492 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.232493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:05.233116 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:05.233118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.233452 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.233454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:05.233513 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:05.233519 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 23:11:05.253198 (+ 19679us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0924 23:11:05.253320 (+ 122us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.212.62:44171: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":80,"threads_started":1}
I20260924 23:11:05.253672 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 18338
2026-09-24T23:11:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (228 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260924 23:11:05.290861 13136 mini_oidc.cc:152] Starting JWKS server
I20260924 23:11:05.291559 13136 webserver.cc:460] Webserver started at https://127.0.0.1:38913/ using document root <none> and password file <none>
I20260924 23:11:05.291610 13136 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 23:11:05.291757 13136 webserver.cc:460] Webserver started at http://127.0.0.1:42541/ using document root <none> and password file <none>
2026-09-24T23:11:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:11:05Z Disabled control of system clock
I20260924 23:11:05.299401 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:46181
--webserver_interface=127.12.212.62
--webserver_port=0
--builtin_ntp_servers=127.12.212.20:42985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:46181
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38913/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260924 23:11:05.381284 18441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:05.381520 18441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:05.381551 18441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:05.382843 18441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:11:05.382893 18441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:05.382910 18441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:05.382926 18441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:11:05.382942 18441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 23:11:05.382958 18441 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 23:11:05.382973 18441 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38913/jwks.json
I20260924 23:11:05.384478 18441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:42985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:46181
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:46181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38913/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:05.384760 18441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:05.385046 18441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:05.390120 18447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:05.390123 18449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:05.390123 18446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:05.390380 18441 server_base.cc:1034] running on GCE node
I20260924 23:11:05.390787 18441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:05.391106 18441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:05.392292 18441 hybrid_clock.cc:648] HybridClock initialized: now 1790291465392274 us; error 30 us; skew 500 ppm
I20260924 23:11:05.393630 18441 webserver.cc:460] Webserver started at http://127.12.212.62:36265/ using document root <none> and password file <none>
I20260924 23:11:05.393899 18441 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:05.393950 18441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:05.394083 18441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:05.395251 18441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "380b5368c19541a18aea04a563ff2b4c"
format_stamp: "Formatted at 2026-09-24 23:11:05 on dist-test-slave-m0z6"
server_key: "8d63612585d8c908d14e8e016ecfdd41"
server_key_iv: "268a0b7738a0d66c5058b27ffb92735b"
server_key_version: "encryptionkey@0"
I20260924 23:11:05.395610 18441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "380b5368c19541a18aea04a563ff2b4c"
format_stamp: "Formatted at 2026-09-24 23:11:05 on dist-test-slave-m0z6"
server_key: "8d63612585d8c908d14e8e016ecfdd41"
server_key_iv: "268a0b7738a0d66c5058b27ffb92735b"
server_key_version: "encryptionkey@0"
I20260924 23:11:05.396904 18441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:11:05.397683 18455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:05.397846 18441 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:05.397930 18441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "380b5368c19541a18aea04a563ff2b4c"
format_stamp: "Formatted at 2026-09-24 23:11:05 on dist-test-slave-m0z6"
server_key: "8d63612585d8c908d14e8e016ecfdd41"
server_key_iv: "268a0b7738a0d66c5058b27ffb92735b"
server_key_version: "encryptionkey@0"
I20260924 23:11:05.397991 18441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:05.419093 18441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:05.419433 18441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:05.419569 18441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:05.423372 18441 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:46181
I20260924 23:11:05.423424 18507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:46181 every 8 connection(s)
I20260924 23:11:05.423818 18441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:11:05.424067 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 18441
I20260924 23:11:05.424166 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:11:05.424379 13136 external_mini_cluster.cc:1428] Setting key a7494b0faff2e322fb64a42b44e5f76b
I20260924 23:11:05.424597 18508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:05.427634 18508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c: Bootstrap starting.
I20260924 23:11:05.428363 18508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:05.428658 18508 log.cc:824] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:05.429466 18508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c: No bootstrap required, opened a new log
I20260924 23:11:05.430593 18511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:05.425333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40248 (local address 127.12.212.62:46181)
0924 23:11:05.425527 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:05.425582 (+ 55us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:05.425610 (+ 28us) server_negotiation.cc:407] Connection header received
0924 23:11:05.426074 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:05.426082 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:05.426151 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:05.426278 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:05.426610 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.427578 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.428623 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.429176 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.429271 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:05.429911 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:05.429916 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:05.429916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:05.430083 (+ 167us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:05.430116 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:05.430127 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:05.430175 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:05.430263 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 23:11:05.430330 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260924 23:11:05.431640 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:11:05.431707 18508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "380b5368c19541a18aea04a563ff2b4c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46181 } }
I20260924 23:11:05.431874 18508 raft_consensus.cc:374] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:05.431906 18508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 380b5368c19541a18aea04a563ff2b4c, State: Initialized, Role: FOLLOWER
I20260924 23:11:05.432029 18508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "380b5368c19541a18aea04a563ff2b4c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46181 } }
I20260924 23:11:05.432101 18508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:11:05.432129 18508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:11:05.432183 18508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:05.432835 18508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "380b5368c19541a18aea04a563ff2b4c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46181 } }
I20260924 23:11:05.432928 18508 leader_election.cc:302] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 380b5368c19541a18aea04a563ff2b4c; no voters:
I20260924 23:11:05.433085 18508 leader_election.cc:288] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:11:05.433152 18513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:05.433295 18513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c [term 1 LEADER]: Becoming Leader. State: Replica: 380b5368c19541a18aea04a563ff2b4c, State: Running, Role: LEADER
I20260924 23:11:05.433334 18508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:11:05.433387 18513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "380b5368c19541a18aea04a563ff2b4c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46181 } }
I20260924 23:11:05.434458 18515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 380b5368c19541a18aea04a563ff2b4c. Latest consensus state: current_term: 1 leader_uuid: "380b5368c19541a18aea04a563ff2b4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "380b5368c19541a18aea04a563ff2b4c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46181 } } }
I20260924 23:11:05.434553 18515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:05.434818 18514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "380b5368c19541a18aea04a563ff2b4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "380b5368c19541a18aea04a563ff2b4c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46181 } } }
I20260924 23:11:05.434880 18514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c [sys.catalog]: This master's current role is: LEADER
W20260924 23:11:05.435072 18527 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:11:05.435111 18527 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:11:05.438825 18528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:11:05.439018 18528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:11:05.440656 18528 catalog_manager.cc:1347] Generated new cluster ID: c30584ef7aef4d388300515954153e1c
I20260924 23:11:05.440719 18528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:11:05.448277 18528 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:11:05.449026 18528 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:11:05.461182 18528 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 380b5368c19541a18aea04a563ff2b4c: Generated new TSK 0
I20260924 23:11:05.461421 18528 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:11:05.473582 18511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:05.469660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40252 (local address 127.12.212.62:46181)
0924 23:11:05.469709 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:05.469711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:05.469789 (+ 78us) server_negotiation.cc:407] Connection header received
0924 23:11:05.469844 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:05.469845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:05.469858 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:05.469950 (+ 92us) server_negotiation.cc:226] Negotiated authn=JWT
0924 23:11:05.470144 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.470972 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.471767 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.472079 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.472144 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:05.472350 (+ 206us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 23:11:05.473363 (+ 1013us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38913/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0924 23:11:05.473501 (+ 138us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40252: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38913/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":26}
W20260924 23:11:05.473656 18511 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40252: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38913/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260924 23:11:05.475598 18537 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:05.469573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:46181 (local address 127.0.0.1:40252)
0924 23:11:05.469719 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:05.469730 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:05.469788 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:05.469981 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:05.469983 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:05.469985 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0924 23:11:05.470046 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:05.470048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.471072 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.471074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:05.471694 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:05.471696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.472182 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.472184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:05.472236 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:05.472242 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 23:11:05.473485 (+ 1243us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38913/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0924 23:11:05.475487 (+ 2002us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.212.62:46181: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38913/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":1719,"thread_start_us":86,"threads_started":1}
I20260924 23:11:05.475997 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 18441
E20260924 23:11:05.486063 18538 webserver.cc:533] Webserver: sslize failed:
E20260924 23:11:05.486155 18538 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-24T23:11:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (222 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-24T23:11:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:11:05Z Disabled control of system clock
I20260924 23:11:05.520625 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:39635
--webserver_interface=127.12.212.62
--webserver_port=0
--builtin_ntp_servers=127.12.212.20:44875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:39635
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790291444018253-13136-0/testchainca.cert with env {}
W20260924 23:11:05.601397 18542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:05.601624 18542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:05.601646 18542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:05.603134 18542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:11:05.603181 18542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:05.603197 18542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:05.603214 18542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:11:05.603229 18542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 23:11:05.603256 18542 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 23:11:05.603279 18542 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260924 23:11:05.605155 18542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:44875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:39635
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:39635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790291444018253-13136-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:05.605422 18542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:05.605629 18542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:05.610369 18548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:05.610365 18547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:05.610422 18542 server_base.cc:1034] running on GCE node
W20260924 23:11:05.610364 18550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:05.610780 18542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:05.611045 18542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:05.612226 18542 hybrid_clock.cc:648] HybridClock initialized: now 1790291465612204 us; error 31 us; skew 500 ppm
I20260924 23:11:05.613509 18542 webserver.cc:460] Webserver started at http://127.12.212.62:41109/ using document root <none> and password file <none>
I20260924 23:11:05.613757 18542 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:05.613804 18542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:05.613922 18542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:05.615082 18542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "050bd75a41014406a964f89532955f75"
format_stamp: "Formatted at 2026-09-24 23:11:05 on dist-test-slave-m0z6"
server_key: "5022a324701b4d17461ebff72dbfd2f8"
server_key_iv: "e8f288b8ffaf72f0c31b9ba20cc48463"
server_key_version: "encryptionkey@0"
I20260924 23:11:05.615424 18542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "050bd75a41014406a964f89532955f75"
format_stamp: "Formatted at 2026-09-24 23:11:05 on dist-test-slave-m0z6"
server_key: "5022a324701b4d17461ebff72dbfd2f8"
server_key_iv: "e8f288b8ffaf72f0c31b9ba20cc48463"
server_key_version: "encryptionkey@0"
I20260924 23:11:05.616720 18542 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 23:11:05.617623 18556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:05.617810 18542 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:11:05.617890 18542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "050bd75a41014406a964f89532955f75"
format_stamp: "Formatted at 2026-09-24 23:11:05 on dist-test-slave-m0z6"
server_key: "5022a324701b4d17461ebff72dbfd2f8"
server_key_iv: "e8f288b8ffaf72f0c31b9ba20cc48463"
server_key_version: "encryptionkey@0"
I20260924 23:11:05.617962 18542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:05.648819 18542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:05.649148 18542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:05.649281 18542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:05.652911 18542 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:39635
I20260924 23:11:05.652973 18608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:39635 every 8 connection(s)
I20260924 23:11:05.653262 18542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:11:05.654242 18609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:05.655030 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 18542
I20260924 23:11:05.655112 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:11:05.655277 13136 external_mini_cluster.cc:1428] Setting key 7a08890e5a31673d6c3495dd0795f8d2
I20260924 23:11:05.657423 18609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75: Bootstrap starting.
I20260924 23:11:05.658125 18609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:05.658394 18609 log.cc:824] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:05.659088 18609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75: No bootstrap required, opened a new log
I20260924 23:11:05.661229 18612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:05.655805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46764 (local address 127.12.212.62:39635)
0924 23:11:05.656015 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:05.656024 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:05.656057 (+ 33us) server_negotiation.cc:407] Connection header received
0924 23:11:05.656561 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:05.656571 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:05.656661 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:05.656865 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:05.657082 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.658350 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.659158 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.659770 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.659903 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:05.660498 (+ 595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:05.660503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:05.660504 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:05.660706 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:05.660752 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:05.660768 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:05.660834 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:05.660888 (+ 54us) server_negotiation.cc:299] Negotiation successful
0924 23:11:05.660979 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260924 23:11:05.661466 18609 raft_consensus.cc:348] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050bd75a41014406a964f89532955f75" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39635 } }
I20260924 23:11:05.661618 18609 raft_consensus.cc:374] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:05.661659 18609 raft_consensus.cc:729] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 050bd75a41014406a964f89532955f75, State: Initialized, Role: FOLLOWER
I20260924 23:11:05.661774 18609 consensus_queue.cc:260] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050bd75a41014406a964f89532955f75" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39635 } }
I20260924 23:11:05.661861 18609 raft_consensus.cc:388] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:11:05.661892 18609 raft_consensus.cc:482] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:11:05.661931 18609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:05.662276 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:11:05.662617 18609 raft_consensus.cc:504] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050bd75a41014406a964f89532955f75" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39635 } }
I20260924 23:11:05.662745 18609 leader_election.cc:302] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 050bd75a41014406a964f89532955f75; no voters:
I20260924 23:11:05.662920 18609 leader_election.cc:288] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:11:05.662976 18614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:05.663149 18614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75 [term 1 LEADER]: Becoming Leader. State: Replica: 050bd75a41014406a964f89532955f75, State: Running, Role: LEADER
I20260924 23:11:05.663172 18609 sys_catalog.cc:564] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:11:05.663245 18614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050bd75a41014406a964f89532955f75" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39635 } }
I20260924 23:11:05.663621 18615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "050bd75a41014406a964f89532955f75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050bd75a41014406a964f89532955f75" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39635 } } }
I20260924 23:11:05.663699 18615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:05.663841 18616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 050bd75a41014406a964f89532955f75. Latest consensus state: current_term: 1 leader_uuid: "050bd75a41014406a964f89532955f75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050bd75a41014406a964f89532955f75" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39635 } } }
I20260924 23:11:05.663910 18616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:05.664032 18622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:11:05.664487 18622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:11:05.665952 18622 catalog_manager.cc:1347] Generated new cluster ID: a807131feb4b495185b77286fb5db9be
I20260924 23:11:05.666002 18622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:11:05.669754 18622 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:11:05.670269 18622 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:11:05.675061 18622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 050bd75a41014406a964f89532955f75: Generated new TSK 0
I20260924 23:11:05.675238 18622 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:11:05.751019 18612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:05.685201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46780 (local address 127.12.212.62:39635)
0924 23:11:05.685247 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:05.685249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:05.685363 (+ 114us) server_negotiation.cc:407] Connection header received
0924 23:11:05.685437 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:05.685438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:05.685452 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:05.685573 (+ 121us) server_negotiation.cc:226] Negotiated authn=JWT
0924 23:11:05.685746 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.686241 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.687083 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.687305 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.687358 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:05.687531 (+ 173us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 23:11:05.750742 (+ 63211us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0924 23:11:05.750940 (+ 198us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46780: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":24}
W20260924 23:11:05.751109 18612 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46780: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260924 23:11:05.751139 18638 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:05.685161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:39635 (local address 127.0.0.1:46780)
0924 23:11:05.685301 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:05.685311 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:05.685372 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:05.685585 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:05.685586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:05.685588 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0924 23:11:05.685648 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:05.685650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.686331 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.686332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:05.687004 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:05.687007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:05.687374 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:05.687375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:05.687432 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:05.687439 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 23:11:05.750889 (+ 63450us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0924 23:11:05.751047 (+ 158us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.212.62:39635: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260924 23:11:05.751482 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 18542
2026-09-24T23:11:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (254 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (89 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (88 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:11:05 dist-test-slave-m0z6 krb5kdc[18650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:11:05 dist-test-slave-m0z6 krb5kdc[18650](info): set up 2 sockets
Sep 24 23:11:05 dist-test-slave-m0z6 krb5kdc[18650](info): commencing operation
krb5kdc: starting...
W20260924 23:11:06.894671 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:11:06 dist-test-slave-m0z6 krb5kdc[18650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:11:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:11:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:11:07.023209 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:44605
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:34329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:44605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:07.106798 18663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:07.107048 18663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:07.107081 18663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:07.108410 18663 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:11:07.108466 18663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:07.108485 18663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:07.108502 18663 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:11:07.108517 18663 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:11:07.110180 18663 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:34329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:44605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:44605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:07.110473 18663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:07.110744 18663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:07.115857 18671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:07.115901 18663 server_base.cc:1034] running on GCE node
W20260924 23:11:07.115857 18669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:07.115913 18668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:07.116362 18663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:07.116693 18663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:07.117856 18663 hybrid_clock.cc:648] HybridClock initialized: now 1790291467117829 us; error 38 us; skew 500 ppm
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[18650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291467, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:07.123977 18663 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:11:07.124524 18663 webserver.cc:460] Webserver started at http://127.12.212.62:34991/ using document root <none> and password file <none>
I20260924 23:11:07.124761 18663 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:07.124814 18663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:07.124979 18663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:07.126062 18663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "6cfd81a3c2ef42e4b5f46496530ec355"
format_stamp: "Formatted at 2026-09-24 23:11:07 on dist-test-slave-m0z6"
server_key: "d06dcd8ecb76c0ca78c7f47d635312c5"
server_key_iv: "aa1f2048baa8630a38a3a0670ffdbc4d"
server_key_version: "encryptionkey@0"
I20260924 23:11:07.126415 18663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "6cfd81a3c2ef42e4b5f46496530ec355"
format_stamp: "Formatted at 2026-09-24 23:11:07 on dist-test-slave-m0z6"
server_key: "d06dcd8ecb76c0ca78c7f47d635312c5"
server_key_iv: "aa1f2048baa8630a38a3a0670ffdbc4d"
server_key_version: "encryptionkey@0"
I20260924 23:11:07.127909 18663 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:11:07.128728 18678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:07.128893 18663 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:07.128993 18663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "6cfd81a3c2ef42e4b5f46496530ec355"
format_stamp: "Formatted at 2026-09-24 23:11:07 on dist-test-slave-m0z6"
server_key: "d06dcd8ecb76c0ca78c7f47d635312c5"
server_key_iv: "aa1f2048baa8630a38a3a0670ffdbc4d"
server_key_version: "encryptionkey@0"
I20260924 23:11:07.129073 18663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:07.151036 18663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:07.153331 18663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:07.153497 18663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:07.157613 18663 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:44605
I20260924 23:11:07.157647 18730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:44605 every 8 connection(s)
I20260924 23:11:07.157995 18663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:11:07.158825 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 18663
I20260924 23:11:07.158964 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:11:07.158995 18731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:07.159162 13136 external_mini_cluster.cc:1428] Setting key fa47e7a4e15ceae052edde57497938ef
I20260924 23:11:07.162410 18731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355: Bootstrap starting.
I20260924 23:11:07.163180 18731 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:07.163532 18731 log.cc:824] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:07.164379 18731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355: No bootstrap required, opened a new log
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[18650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:07.166983 18731 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cfd81a3c2ef42e4b5f46496530ec355" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 44605 } }
I20260924 23:11:07.167194 18731 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:07.167253 18731 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cfd81a3c2ef42e4b5f46496530ec355, State: Initialized, Role: FOLLOWER
I20260924 23:11:07.167407 18731 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cfd81a3c2ef42e4b5f46496530ec355" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 44605 } }
I20260924 23:11:07.167482 18731 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:11:07.167512 18731 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:11:07.167559 18731 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:07.168131 18731 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cfd81a3c2ef42e4b5f46496530ec355" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 44605 } }
I20260924 23:11:07.168233 18731 leader_election.cc:302] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cfd81a3c2ef42e4b5f46496530ec355; no voters:
I20260924 23:11:07.168414 18731 leader_election.cc:288] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:11:07.168478 18736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:07.168678 18736 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355 [term 1 LEADER]: Becoming Leader. State: Replica: 6cfd81a3c2ef42e4b5f46496530ec355, State: Running, Role: LEADER
I20260924 23:11:07.168684 18731 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:11:07.168809 18736 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cfd81a3c2ef42e4b5f46496530ec355" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 44605 } }
I20260924 23:11:07.169221 18737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cfd81a3c2ef42e4b5f46496530ec355" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cfd81a3c2ef42e4b5f46496530ec355" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 44605 } } }
I20260924 23:11:07.169371 18737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:07.169219 18736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cfd81a3c2ef42e4b5f46496530ec355. Latest consensus state: current_term: 1 leader_uuid: "6cfd81a3c2ef42e4b5f46496530ec355" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cfd81a3c2ef42e4b5f46496530ec355" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 44605 } } }
I20260924 23:11:07.169611 18745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:11:07.170245 18745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:11:07.169632 18736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:07.170857 18734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:07.159726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39476 (local address 127.12.212.62:44605)
0924 23:11:07.159899 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:07.159908 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:07.159934 (+ 26us) server_negotiation.cc:407] Connection header received
0924 23:11:07.160614 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:07.160624 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:07.160685 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:07.160815 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:07.161685 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.162683 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.163748 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.164294 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.164381 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:07.167169 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:07.167174 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:07.167174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:07.167187 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:07.169192 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:07.169573 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:07.169576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:07.169577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:07.169685 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:07.169897 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:07.169898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:07.169898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:07.170214 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:07.170356 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:07.170481 (+ 125us) server_negotiation.cc:299] Negotiation successful
0924 23:11:07.170619 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260924 23:11:07.172715 18745 catalog_manager.cc:1347] Generated new cluster ID: 50c7b976321d4dee8b1e6dc32f1cb4ef
I20260924 23:11:07.172765 18745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:11:07.181632 18745 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:11:07.182264 18745 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:11:07.185422 18745 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cfd81a3c2ef42e4b5f46496530ec355: Generated new TSK 0
I20260924 23:11:07.185601 18745 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
I20260924 23:11:07.219542 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:44605
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:34329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:07.322222 18757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:07.322443 18757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:07.322468 18757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:07.323729 18757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:07.323781 18757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:07.323807 18757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:11:07.325412 18757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:34329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:44605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:07.325627 18757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:07.325832 18757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:07.330813 18762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:07.331027 18757 server_base.cc:1034] running on GCE node
W20260924 23:11:07.330825 18763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:07.330813 18765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:07.331316 18757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:07.331609 18757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:07.332787 18757 hybrid_clock.cc:648] HybridClock initialized: now 1790291467332771 us; error 32 us; skew 500 ppm
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[18650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291467, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:07.335476 18757 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:11:07.335898 18757 webserver.cc:460] Webserver started at http://127.12.212.1:36511/ using document root <none> and password file <none>
I20260924 23:11:07.336100 18757 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:07.336143 18757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:07.336236 18757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:07.337256 18757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "626b7f12cb4c4a1ba95f44e87c368a44"
format_stamp: "Formatted at 2026-09-24 23:11:07 on dist-test-slave-m0z6"
server_key: "0712f8ba2b07fc07c6b449cf9177ac6a"
server_key_iv: "69f6e01d848ed61696119088795ef7d6"
server_key_version: "encryptionkey@0"
I20260924 23:11:07.337559 18757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "626b7f12cb4c4a1ba95f44e87c368a44"
format_stamp: "Formatted at 2026-09-24 23:11:07 on dist-test-slave-m0z6"
server_key: "0712f8ba2b07fc07c6b449cf9177ac6a"
server_key_iv: "69f6e01d848ed61696119088795ef7d6"
server_key_version: "encryptionkey@0"
I20260924 23:11:07.338902 18757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:11:07.339709 18772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:07.339917 18757 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:07.340009 18757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "626b7f12cb4c4a1ba95f44e87c368a44"
format_stamp: "Formatted at 2026-09-24 23:11:07 on dist-test-slave-m0z6"
server_key: "0712f8ba2b07fc07c6b449cf9177ac6a"
server_key_iv: "69f6e01d848ed61696119088795ef7d6"
server_key_version: "encryptionkey@0"
I20260924 23:11:07.340098 18757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:07.359902 18757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:07.361986 18757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:07.362106 18757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:07.362358 18757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:07.362741 18757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:07.362778 18757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:07.362800 18757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:07.362811 18757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:07.368353 18757 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:39483
I20260924 23:11:07.368438 18885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:39483 every 8 connection(s)
I20260924 23:11:07.368769 18757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[18650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291467, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:07.375661 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 18757
I20260924 23:11:07.375761 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:11:07.376021 13136 external_mini_cluster.cc:1428] Setting key 2d38d290012dd62dec9e63e5bb5d8640
I20260924 23:11:07.378629 18734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:07.369653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:56009 (local address 127.12.212.62:44605)
0924 23:11:07.369715 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:07.369717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:07.369914 (+ 197us) server_negotiation.cc:407] Connection header received
0924 23:11:07.370955 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:07.370957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:07.370971 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:07.371020 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:07.371678 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.372181 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.373229 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.373445 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.373513 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:07.375619 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:07.375620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:07.375621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:07.375628 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:07.377075 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:07.377471 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:07.377472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:07.377472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:07.377538 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:07.377827 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:07.377829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:07.377829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:07.377967 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:07.378028 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:07.378439 (+ 411us) server_negotiation.cc:299] Negotiation successful
0924 23:11:07.378554 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 23:11:07.378783 18888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:07.369631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:44605 (local address 127.12.212.1:56009)
0924 23:11:07.369771 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:07.369828 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:07.370835 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:07.371110 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:07.371115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:07.371374 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:07.371570 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:07.371575 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.372273 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.372274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:07.373131 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:07.373134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.373525 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.373527 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:07.373650 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:07.374134 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:07.374142 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:07.375527 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:07.377148 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:07.377155 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:07.377157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:07.377399 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:07.377598 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:07.377598 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:07.377599 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:07.377729 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:07.378037 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:07.378042 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:07.378220 (+ 178us) client_negotiation.cc:769] Sending connection context
0924 23:11:07.378334 (+ 114us) client_negotiation.cc:240] Negotiation successful
0924 23:11:07.378521 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":64,"threads_started":1}
I20260924 23:11:07.379314 18886 heartbeater.cc:347] Connected to a master server at 127.12.212.62:44605
I20260924 23:11:07.379413 18886 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:07.379604 18886 heartbeater.cc:511] Master 127.12.212.62:44605 requested a full tablet report, sending...
I20260924 23:11:07.380144 18695 ts_manager.cc:194] Registered new tserver with Master: 626b7f12cb4c4a1ba95f44e87c368a44 (127.12.212.1:39483)
I20260924 23:11:07.380774 18695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:56009
WARNING: no policy specified for kudu/127.12.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
I20260924 23:11:07.399533 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:44605
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:34329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:07.482138 18891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:07.482383 18891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:07.482415 18891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:07.483796 18891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:07.483850 18891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:07.483878 18891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:11:07.485452 18891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:34329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:44605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:07.485682 18891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:07.485874 18891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:07.491106 18896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:07.491110 18899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:07.491110 18897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:07.491325 18891 server_base.cc:1034] running on GCE node
I20260924 23:11:07.491694 18891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:07.491969 18891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:07.493173 18891 hybrid_clock.cc:648] HybridClock initialized: now 1790291467493156 us; error 33 us; skew 500 ppm
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[18650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291467, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:07.496053 18891 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:11:07.496531 18891 webserver.cc:460] Webserver started at http://127.12.212.2:43963/ using document root <none> and password file <none>
I20260924 23:11:07.496744 18891 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:07.496788 18891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:07.496889 18891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:07.497977 18891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "4161ebc7519d42448d1bc9ac5f53b8a1"
format_stamp: "Formatted at 2026-09-24 23:11:07 on dist-test-slave-m0z6"
server_key: "aaf583e71ec8ca2796cf6ae46d707d15"
server_key_iv: "13aa9000ee6ce2f70de5398be84ae04c"
server_key_version: "encryptionkey@0"
I20260924 23:11:07.498288 18891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "4161ebc7519d42448d1bc9ac5f53b8a1"
format_stamp: "Formatted at 2026-09-24 23:11:07 on dist-test-slave-m0z6"
server_key: "aaf583e71ec8ca2796cf6ae46d707d15"
server_key_iv: "13aa9000ee6ce2f70de5398be84ae04c"
server_key_version: "encryptionkey@0"
I20260924 23:11:07.499822 18891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:11:07.500789 18906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:07.500999 18891 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:07.501120 18891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "4161ebc7519d42448d1bc9ac5f53b8a1"
format_stamp: "Formatted at 2026-09-24 23:11:07 on dist-test-slave-m0z6"
server_key: "aaf583e71ec8ca2796cf6ae46d707d15"
server_key_iv: "13aa9000ee6ce2f70de5398be84ae04c"
server_key_version: "encryptionkey@0"
I20260924 23:11:07.501189 18891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:07.559268 18891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:07.561623 18891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:07.561746 18891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:07.561954 18891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:07.562230 18891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:07.562265 18891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:07.562295 18891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:07.562314 18891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:07.567932 18891 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:39517
I20260924 23:11:07.568008 19019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:39517 every 8 connection(s)
I20260924 23:11:07.568352 18891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:11:07.574978 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 18891
I20260924 23:11:07.575083 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:11:07.575248 13136 external_mini_cluster.cc:1428] Setting key 80dfa9cd34e2e00dbce540ce475a573f
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[18650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291467, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:07.580152 18734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:07.569270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:53945 (local address 127.12.212.62:44605)
0924 23:11:07.569317 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:07.569319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:07.569615 (+ 296us) server_negotiation.cc:407] Connection header received
0924 23:11:07.570806 (+ 1191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:07.570808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:07.570820 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:07.570859 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:07.571616 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.572125 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.573271 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.573520 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.573592 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:07.576455 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:07.576457 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:07.576458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:07.576466 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:07.578098 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:07.578773 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:07.578774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:07.578775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:07.578848 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:07.579190 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:07.579191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:07.579191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:07.579360 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:07.579426 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:07.579967 (+ 541us) server_negotiation.cc:299] Negotiation successful
0924 23:11:07.580075 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 23:11:07.580343 19022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:07.569233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:44605 (local address 127.12.212.2:53945)
0924 23:11:07.569480 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:07.569526 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:07.570653 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:07.570943 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:07.570947 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:07.571237 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:07.571506 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:07.571511 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.572243 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.572244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:07.573126 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:07.573129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.573621 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.573622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:07.573748 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:07.574317 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:07.574327 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:07.576314 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:07.578218 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:07.578232 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:07.578235 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:07.578626 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:07.578927 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:07.578928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:07.578928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:07.579121 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:07.579443 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:07.579452 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:07.579657 (+ 205us) client_negotiation.cc:769] Sending connection context
0924 23:11:07.579779 (+ 122us) client_negotiation.cc:240] Negotiation successful
0924 23:11:07.580004 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
I20260924 23:11:07.580883 19020 heartbeater.cc:347] Connected to a master server at 127.12.212.62:44605
I20260924 23:11:07.581017 19020 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:07.581238 19020 heartbeater.cc:511] Master 127.12.212.62:44605 requested a full tablet report, sending...
I20260924 23:11:07.581694 18695 ts_manager.cc:194] Registered new tserver with Master: 4161ebc7519d42448d1bc9ac5f53b8a1 (127.12.212.2:39517)
I20260924 23:11:07.582074 18695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:53945
WARNING: no policy specified for kudu/127.12.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
I20260924 23:11:07.601929 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:44605
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:34329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:07.684481 19025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:07.684725 19025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:07.684758 19025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:07.686125 19025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:07.686174 19025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:07.686213 19025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:11:07.687832 19025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:34329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:44605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:07.688143 19025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:07.688359 19025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:07.693660 19030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:07.693665 19031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:07.693665 19033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:07.693698 19025 server_base.cc:1034] running on GCE node
I20260924 23:11:07.694146 19025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:07.694419 19025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:07.695617 19025 hybrid_clock.cc:648] HybridClock initialized: now 1790291467695595 us; error 33 us; skew 500 ppm
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[18650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291467, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:07.698402 19025 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:11:07.698987 19025 webserver.cc:460] Webserver started at http://127.12.212.3:43355/ using document root <none> and password file <none>
I20260924 23:11:07.699226 19025 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:07.699278 19025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:07.699412 19025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:07.700495 19025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "cbcbe752c78e43fcbb522d802f53b8fe"
format_stamp: "Formatted at 2026-09-24 23:11:07 on dist-test-slave-m0z6"
server_key: "1f7e6bec7b8b8e43c9643c26e0637ebf"
server_key_iv: "f74f97e33cf0799736d6382e3886f3b5"
server_key_version: "encryptionkey@0"
I20260924 23:11:07.700865 19025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "cbcbe752c78e43fcbb522d802f53b8fe"
format_stamp: "Formatted at 2026-09-24 23:11:07 on dist-test-slave-m0z6"
server_key: "1f7e6bec7b8b8e43c9643c26e0637ebf"
server_key_iv: "f74f97e33cf0799736d6382e3886f3b5"
server_key_version: "encryptionkey@0"
I20260924 23:11:07.702324 19025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:11:07.703253 19040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:07.703526 19025 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 23:11:07.703624 19025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "cbcbe752c78e43fcbb522d802f53b8fe"
format_stamp: "Formatted at 2026-09-24 23:11:07 on dist-test-slave-m0z6"
server_key: "1f7e6bec7b8b8e43c9643c26e0637ebf"
server_key_iv: "f74f97e33cf0799736d6382e3886f3b5"
server_key_version: "encryptionkey@0"
I20260924 23:11:07.703697 19025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:07.716301 19025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:07.718351 19025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:07.718506 19025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:07.718786 19025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:07.719125 19025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:07.719161 19025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:07.719190 19025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:07.719208 19025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:07.725359 19025 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:41969
I20260924 23:11:07.725389 19153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:41969 every 8 connection(s)
I20260924 23:11:07.725857 19025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:11:07.726905 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 19025
I20260924 23:11:07.727006 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:11:07.727175 13136 external_mini_cluster.cc:1428] Setting key 355441c651a1a469e34e160cca495495
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[18650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291467, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:07.736426 18734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:07.726787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:53495 (local address 127.12.212.62:44605)
0924 23:11:07.726861 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:07.726863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:07.727079 (+ 216us) server_negotiation.cc:407] Connection header received
0924 23:11:07.728423 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:07.728424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:07.728438 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:07.728484 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:07.729184 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.729854 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.730966 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.731168 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.731224 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:07.733696 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:07.733699 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:07.733699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:07.733708 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:07.734988 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:07.735427 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:07.735428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:07.735428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:07.735493 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:07.735760 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:07.735760 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:07.735761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:07.735881 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:07.735942 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:07.736237 (+ 295us) server_negotiation.cc:299] Negotiation successful
0924 23:11:07.736325 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 23:11:07.736649 19156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:07.726759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:44605 (local address 127.12.212.3:53495)
0924 23:11:07.726946 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:07.726996 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:07.728315 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:07.728579 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:07.728583 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:07.728852 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:07.729083 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:07.729089 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.729980 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.729981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:07.730884 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:07.730887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.731249 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.731250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:07.731402 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:07.732022 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:07.732036 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:07.733581 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:07.735075 (+ 1494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:07.735086 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:07.735089 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:07.735347 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:07.735558 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:07.735559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:07.735559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:07.735695 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:07.735946 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:07.735954 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:07.736111 (+ 157us) client_negotiation.cc:769] Sending connection context
0924 23:11:07.736206 (+ 95us) client_negotiation.cc:240] Negotiation successful
0924 23:11:07.736415 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":83,"threads_started":1}
I20260924 23:11:07.737221 19154 heartbeater.cc:347] Connected to a master server at 127.12.212.62:44605
I20260924 23:11:07.737336 19154 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:07.737511 19154 heartbeater.cc:511] Master 127.12.212.62:44605 requested a full tablet report, sending...
I20260924 23:11:07.737980 18695 ts_manager.cc:194] Registered new tserver with Master: cbcbe752c78e43fcbb522d802f53b8fe (127.12.212.3:41969)
I20260924 23:11:07.738355 18695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:53495
I20260924 23:11:07.739238 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/test-admin.keytab.
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[18650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:07.756580 13136 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260924 23:11:07.758261 19161 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:07.757438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43912 (local address 127.12.212.1:39483)
0924 23:11:07.757631 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:07.757636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:07.757644 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:11:07.757668 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:07.757670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:07.757684 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:07.757749 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:07.758177 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43912: BlockingRecv error: recv got EOF from 127.0.0.1:43912 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":74,"threads_started":1}
W20260924 23:11:07.758440 13136 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/krb5cc' of type FILE
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[18650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:07.759491 13136 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[18650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:07.760407 13136 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[18650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.1@KRBTEST.COM
I20260924 23:11:07.769249 19161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:07.760931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43928 (local address 127.12.212.1:39483)
0924 23:11:07.760980 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:07.760982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:07.761007 (+ 25us) server_negotiation.cc:407] Connection header received
0924 23:11:07.761064 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:07.761065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:07.761074 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:07.761126 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:07.761528 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.762269 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.763085 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.763439 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.763509 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:07.765926 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:07.765936 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:07.765937 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:07.765944 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:07.767767 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:07.768204 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:07.768210 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:07.768211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:07.768288 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:07.768612 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:07.768614 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:07.768614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:07.768786 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:07.768923 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:07.769055 (+ 132us) server_negotiation.cc:299] Negotiation successful
0924 23:11:07.769132 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260924 23:11:07.771626 19161 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:07.770963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43930 (local address 127.12.212.1:39483)
0924 23:11:07.771027 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:07.771029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:07.771083 (+ 54us) server_negotiation.cc:407] Connection header received
0924 23:11:07.771123 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:07.771124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:07.771133 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:07.771185 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:07.771570 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43930: BlockingRecv error: recv got EOF from 127.0.0.1:43930 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260924 23:11:07.771765 13136 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/krb5cc' of type FILE
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[18650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:07.772903 13136 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[18650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:07.773779 13136 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[18650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.1@KRBTEST.COM
I20260924 23:11:07.781370 19161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:07.774415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43938 (local address 127.12.212.1:39483)
0924 23:11:07.774520 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:07.774522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:07.774530 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:11:07.774569 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:07.774570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:07.774581 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:07.774631 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:07.775083 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.775575 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.776304 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.776507 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.776566 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:07.778754 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:07.778757 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:07.778757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:07.778765 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:07.780227 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:07.780611 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:07.780613 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:07.780613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:07.780675 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:07.780859 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:07.780860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:07.780860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:07.780965 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:07.781053 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:07.781184 (+ 131us) server_negotiation.cc:299] Negotiation successful
0924 23:11:07.781271 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260924 23:11:07.783094 19161 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:07.782406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43948 (local address 127.12.212.1:39483)
0924 23:11:07.782475 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:07.782477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:07.782541 (+ 64us) server_negotiation.cc:407] Connection header received
0924 23:11:07.782599 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:07.782600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:07.782609 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:07.782653 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:07.783038 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43948: BlockingRecv error: recv got EOF from 127.0.0.1:43948 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260924 23:11:07.783174 13136 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790291444018253-13136-0/krb5kdc/krb5cc' of type FILE
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[18650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:07.784219 13136 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[18650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:07.785394 13136 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[18650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.1@KRBTEST.COM
I20260924 23:11:07.792393 19161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:07.785969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43958 (local address 127.12.212.1:39483)
0924 23:11:07.786049 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:07.786051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:07.786065 (+ 14us) server_negotiation.cc:407] Connection header received
0924 23:11:07.786142 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:07.786143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:07.786153 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:07.786200 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:07.786570 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.787143 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.787838 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:07.788054 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:07.788108 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:07.790119 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:07.790120 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:07.790121 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:07.790128 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:07.791436 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:07.791738 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:07.791738 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:07.791739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:07.791791 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:07.791925 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:07.791926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:07.791926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:07.792018 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:07.792093 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:07.792194 (+ 101us) server_negotiation.cc:299] Negotiation successful
0924 23:11:07.792250 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 23:11:07.793187 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 18757
I20260924 23:11:07.796872 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 18891
I20260924 23:11:07.800693 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 19025
I20260924 23:11:07.804739 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 18663
2026-09-24T23:11:07Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1876 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[19174](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[19174](info): set up 2 sockets
Sep 24 23:11:07 dist-test-slave-m0z6 krb5kdc[19174](info): commencing operation
krb5kdc: starting...
W20260924 23:11:08.783257 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.960s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:11:08 dist-test-slave-m0z6 krb5kdc[19174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:11:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:11:08Z Disabled control of system clock
WARNING: no policy specified for oryx/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.62.keytab.
Entry for principal oryx/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.62.keytab.
I20260924 23:11:08.906899 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:37579
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.62.keytab
--principal=oryx/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:37579
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:08.989845 19187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:08.990082 19187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:08.990106 19187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:08.991407 19187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:11:08.991466 19187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:08.991482 19187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:08.991493 19187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:11:08.991504 19187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:11:08.993197 19187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:37579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.62.keytab
--principal=oryx/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:37579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:08.993515 19187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:08.993713 19187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:08.999733 19195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:08.999737 19193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:08.999773 19187 server_base.cc:1034] running on GCE node
W20260924 23:11:08.999737 19192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:09.000334 19187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:09.000672 19187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:09.001866 19187 hybrid_clock.cc:648] HybridClock initialized: now 1790291469001850 us; error 31 us; skew 500 ppm
Sep 24 23:11:08 dist-test-slave-m0z6 krb5kdc[19174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291468, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:09.004483 19187 init.cc:377] Logged in from keytab as oryx/127.12.212.62@KRBTEST.COM (short username oryx)
I20260924 23:11:09.004981 19187 webserver.cc:460] Webserver started at http://127.12.212.62:42097/ using document root <none> and password file <none>
I20260924 23:11:09.005190 19187 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:09.005239 19187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:09.005370 19187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:09.006445 19187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "838e7664b2b344b980d6324a8a2e3ec4"
format_stamp: "Formatted at 2026-09-24 23:11:08 on dist-test-slave-m0z6"
server_key: "31127a3b2b002743d541d17c5a6e58cf"
server_key_iv: "b9814764cdf615f45b7aed89aed2fb13"
server_key_version: "encryptionkey@0"
I20260924 23:11:09.006829 19187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "838e7664b2b344b980d6324a8a2e3ec4"
format_stamp: "Formatted at 2026-09-24 23:11:08 on dist-test-slave-m0z6"
server_key: "31127a3b2b002743d541d17c5a6e58cf"
server_key_iv: "b9814764cdf615f45b7aed89aed2fb13"
server_key_version: "encryptionkey@0"
I20260924 23:11:09.008226 19187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:11:09.009198 19202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:09.009514 19187 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:11:09.009624 19187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "838e7664b2b344b980d6324a8a2e3ec4"
format_stamp: "Formatted at 2026-09-24 23:11:08 on dist-test-slave-m0z6"
server_key: "31127a3b2b002743d541d17c5a6e58cf"
server_key_iv: "b9814764cdf615f45b7aed89aed2fb13"
server_key_version: "encryptionkey@0"
I20260924 23:11:09.009707 19187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:09.034226 19187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:09.037002 19187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:09.037168 19187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:09.041226 19187 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:37579
I20260924 23:11:09.041272 19254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:37579 every 8 connection(s)
I20260924 23:11:09.041635 19187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:11:09.042236 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 19187
I20260924 23:11:09.042412 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:11:09.042605 13136 external_mini_cluster.cc:1428] Setting key 1b385011012a0d69ff6bfb56704472e5
I20260924 23:11:09.042733 19255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:09.046120 19255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4: Bootstrap starting.
I20260924 23:11:09.046819 19255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:09.047283 19255 log.cc:824] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:09.048177 19255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4: No bootstrap required, opened a new log
Sep 24 23:11:09 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM
I20260924 23:11:09.050606 19255 raft_consensus.cc:348] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } }
I20260924 23:11:09.050837 19255 raft_consensus.cc:374] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:09.050874 19255 raft_consensus.cc:729] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838e7664b2b344b980d6324a8a2e3ec4, State: Initialized, Role: FOLLOWER
I20260924 23:11:09.051026 19255 consensus_queue.cc:260] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } }
I20260924 23:11:09.051129 19255 raft_consensus.cc:388] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:11:09.051163 19255 raft_consensus.cc:482] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:11:09.051215 19255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:09.052027 19255 raft_consensus.cc:504] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } }
I20260924 23:11:09.052145 19255 leader_election.cc:302] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 838e7664b2b344b980d6324a8a2e3ec4; no voters:
I20260924 23:11:09.052347 19255 leader_election.cc:288] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:11:09.052400 19260 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:09.052522 19260 raft_consensus.cc:686] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [term 1 LEADER]: Becoming Leader. State: Replica: 838e7664b2b344b980d6324a8a2e3ec4, State: Running, Role: LEADER
I20260924 23:11:09.052605 19260 consensus_queue.cc:237] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } }
I20260924 23:11:09.052640 19255 sys_catalog.cc:564] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:11:09.052980 19260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "838e7664b2b344b980d6324a8a2e3ec4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } }
I20260924 23:11:09.053061 19260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:09.053175 19260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 838e7664b2b344b980d6324a8a2e3ec4. Latest consensus state: current_term: 1 leader_uuid: "838e7664b2b344b980d6324a8a2e3ec4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } }
I20260924 23:11:09.053226 19260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: This master's current role is: LEADER
W20260924 23:11:09.054557 19276 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:11:09.054618 19276 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:11:09.054672 19266 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:11:09.054890 19266 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:11:09.055212 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.043233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35034 (local address 127.12.212.62:37579)
0924 23:11:09.043462 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:09.043472 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:09.043507 (+ 35us) server_negotiation.cc:407] Connection header received
0924 23:11:09.043942 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:09.043952 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:09.044022 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:09.044178 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:09.044864 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.046212 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.047267 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.048008 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.048135 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.050946 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:09.050951 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:09.050952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:09.050966 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:09.053222 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.053767 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.053772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.053773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.053903 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.054186 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.054187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.054188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.054470 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:09.054605 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:09.054821 (+ 216us) server_negotiation.cc:299] Negotiation successful
0924 23:11:09.055053 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":112,"threads_started":1}
I20260924 23:11:09.056577 19266 catalog_manager.cc:1347] Generated new cluster ID: 95821273abe2409e9ab3091d80765996
I20260924 23:11:09.056628 19266 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:11:09.063998 19266 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:11:09.064595 19266 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:11:09.067234 19266 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4: Generated new TSK 0
I20260924 23:11:09.067472 19266 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.1.keytab.
Entry for principal oryx/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.1.keytab.
I20260924 23:11:09.101867 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:37579
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.1.keytab
--principal=oryx/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:09.187908 19281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:09.188135 19281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:09.188159 19281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:09.189519 19281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:09.189579 19281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:09.189607 19281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:11:09.191277 19281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.1.keytab
--principal=oryx/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:37579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:09.191501 19281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:09.191697 19281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:09.197254 19287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:09.197230 19289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:09.197228 19286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:09.197588 19281 server_base.cc:1034] running on GCE node
I20260924 23:11:09.197798 19281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:09.198048 19281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:09.199223 19281 hybrid_clock.cc:648] HybridClock initialized: now 1790291469199198 us; error 38 us; skew 500 ppm
Sep 24 23:11:09 dist-test-slave-m0z6 krb5kdc[19174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291469, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:09.201884 19281 init.cc:377] Logged in from keytab as oryx/127.12.212.1@KRBTEST.COM (short username oryx)
I20260924 23:11:09.202350 19281 webserver.cc:460] Webserver started at http://127.12.212.1:36737/ using document root <none> and password file <none>
I20260924 23:11:09.202553 19281 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:09.202603 19281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:09.202776 19281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:09.203871 19281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "5816efe62c724db78e74db9e416e81ea"
format_stamp: "Formatted at 2026-09-24 23:11:09 on dist-test-slave-m0z6"
server_key: "8502ef28c88ae01c6a22a909e374d362"
server_key_iv: "17b7a2c65d93700210a670c01460c194"
server_key_version: "encryptionkey@0"
I20260924 23:11:09.204213 19281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "5816efe62c724db78e74db9e416e81ea"
format_stamp: "Formatted at 2026-09-24 23:11:09 on dist-test-slave-m0z6"
server_key: "8502ef28c88ae01c6a22a909e374d362"
server_key_iv: "17b7a2c65d93700210a670c01460c194"
server_key_version: "encryptionkey@0"
I20260924 23:11:09.205623 19281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:11:09.206552 19296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:09.206768 19281 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:09.206857 19281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "5816efe62c724db78e74db9e416e81ea"
format_stamp: "Formatted at 2026-09-24 23:11:09 on dist-test-slave-m0z6"
server_key: "8502ef28c88ae01c6a22a909e374d362"
server_key_iv: "17b7a2c65d93700210a670c01460c194"
server_key_version: "encryptionkey@0"
I20260924 23:11:09.206918 19281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:09.217571 19281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:09.219781 19281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:09.219946 19281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:09.220147 19281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:09.220424 19281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:09.220459 19281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:09.220481 19281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:09.220500 19281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:09.226264 19281 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:33293
I20260924 23:11:09.226398 19409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:33293 every 8 connection(s)
I20260924 23:11:09.226675 19281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:11:09.227046 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 19281
I20260924 23:11:09.227149 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:11:09.227432 13136 external_mini_cluster.cc:1428] Setting key af28c502e2a0ca3640088323c95ef948
Sep 24 23:11:09 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291469, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.12.212.2@KRBTEST.COM; defaulting to no policy
I20260924 23:11:09.236320 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.227523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:45457 (local address 127.12.212.62:37579)
0924 23:11:09.227563 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:09.227566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:09.227838 (+ 272us) server_negotiation.cc:407] Connection header received
0924 23:11:09.228439 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:09.228440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:09.228451 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:09.228513 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:09.229465 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.229965 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.231000 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.231209 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.231266 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.233613 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:09.233614 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:09.233615 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:09.233623 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:09.234980 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.235397 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.235398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.235399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.235457 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.235690 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.235691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.235691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.235803 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:09.235860 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:09.236132 (+ 272us) server_negotiation.cc:299] Negotiation successful
0924 23:11:09.236223 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 23:11:09.236495 19412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.227501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.1:45457)
0924 23:11:09.227675 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:09.227726 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:09.228320 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:09.228573 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:09.228577 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:09.229139 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:09.229367 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:09.229372 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.230059 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.230060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:09.230921 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:09.230924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.231283 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.231284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:09.231400 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.231981 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:09.231990 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:09.233516 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:09.235058 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:09.235065 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:09.235067 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:09.235331 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:09.235520 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:09.235521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:09.235521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:09.235631 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:09.235863 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:09.235870 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:09.236008 (+ 138us) client_negotiation.cc:769] Sending connection context
0924 23:11:09.236101 (+ 93us) client_negotiation.cc:240] Negotiation successful
0924 23:11:09.236263 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260924 23:11:09.237010 19410 heartbeater.cc:347] Connected to a master server at 127.12.212.62:37579
I20260924 23:11:09.237113 19410 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:09.237308 19410 heartbeater.cc:511] Master 127.12.212.62:37579 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.212.2@KRBTEST.COM" created.
I20260924 23:11:09.237922 19219 ts_manager.cc:194] Registered new tserver with Master: 5816efe62c724db78e74db9e416e81ea (127.12.212.1:33293)
I20260924 23:11:09.238813 19219 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:45457
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.2.keytab.
Entry for principal oryx/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.2.keytab.
I20260924 23:11:09.254299 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:37579
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.2.keytab
--principal=oryx/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:09.337463 19415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:09.337675 19415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:09.337699 19415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:09.338882 19415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:09.338934 19415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:09.338972 19415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:11:09.340452 19415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.2.keytab
--principal=oryx/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:37579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:09.340654 19415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:09.340848 19415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:09.346657 19420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:09.346664 19415 server_base.cc:1034] running on GCE node
W20260924 23:11:09.346755 19421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:09.346944 19423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:09.347270 19415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:09.347554 19415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:09.348752 19415 hybrid_clock.cc:648] HybridClock initialized: now 1790291469348732 us; error 29 us; skew 500 ppm
Sep 24 23:11:09 dist-test-slave-m0z6 krb5kdc[19174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291469, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:09.351506 19415 init.cc:377] Logged in from keytab as oryx/127.12.212.2@KRBTEST.COM (short username oryx)
I20260924 23:11:09.351958 19415 webserver.cc:460] Webserver started at http://127.12.212.2:42699/ using document root <none> and password file <none>
I20260924 23:11:09.352180 19415 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:09.352229 19415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:09.352362 19415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:09.353574 19415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "247c95b3bbae46d395cb9822ac4f9225"
format_stamp: "Formatted at 2026-09-24 23:11:09 on dist-test-slave-m0z6"
server_key: "063ebb8fe7be4db56c8b1f1b4e801bd2"
server_key_iv: "20b02f5f4a88f175c8ebdec7a5aa2ff4"
server_key_version: "encryptionkey@0"
I20260924 23:11:09.353927 19415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "247c95b3bbae46d395cb9822ac4f9225"
format_stamp: "Formatted at 2026-09-24 23:11:09 on dist-test-slave-m0z6"
server_key: "063ebb8fe7be4db56c8b1f1b4e801bd2"
server_key_iv: "20b02f5f4a88f175c8ebdec7a5aa2ff4"
server_key_version: "encryptionkey@0"
I20260924 23:11:09.355290 19415 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 23:11:09.356101 19430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:09.356323 19415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:09.356406 19415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "247c95b3bbae46d395cb9822ac4f9225"
format_stamp: "Formatted at 2026-09-24 23:11:09 on dist-test-slave-m0z6"
server_key: "063ebb8fe7be4db56c8b1f1b4e801bd2"
server_key_iv: "20b02f5f4a88f175c8ebdec7a5aa2ff4"
server_key_version: "encryptionkey@0"
I20260924 23:11:09.356511 19415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:09.372963 19415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:09.375296 19415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:09.375480 19415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:09.375722 19415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:09.376030 19415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:09.376067 19415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:09.376097 19415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:09.376113 19415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:09.381639 19415 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:38875
I20260924 23:11:09.381739 19543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:38875 every 8 connection(s)
I20260924 23:11:09.382015 19415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
Sep 24 23:11:09 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291469, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM
I20260924 23:11:09.389825 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 19415
I20260924 23:11:09.389945 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:11:09.390156 13136 external_mini_cluster.cc:1428] Setting key 2c1491a5cd94679f46a1353164aa31f8
I20260924 23:11:09.391955 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.382824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:60383 (local address 127.12.212.62:37579)
0924 23:11:09.382896 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:09.382898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:09.383065 (+ 167us) server_negotiation.cc:407] Connection header received
0924 23:11:09.383658 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:09.383660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:09.383672 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:09.383720 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:09.384691 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.385195 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.386194 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.386400 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.386458 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.388798 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:09.388800 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:09.388801 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:09.388809 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:09.390291 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.390786 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.390788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.390788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.390861 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.391170 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.391171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.391171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.391321 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:09.391387 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:09.391765 (+ 378us) server_negotiation.cc:299] Negotiation successful
0924 23:11:09.391858 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 23:11:09.392076 19546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.382754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.2:60383)
0924 23:11:09.382940 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:09.382977 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:09.383547 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:09.383803 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:09.383806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:09.384382 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:09.384580 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:09.384585 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.385295 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.385296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:09.386102 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:09.386104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.386478 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.386479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:09.386597 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.387145 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:09.387154 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:09.388683 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:09.390383 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:09.390392 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:09.390395 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:09.390721 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:09.390925 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:09.390926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:09.390926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:09.391079 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:09.391415 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:09.391422 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:09.391610 (+ 188us) client_negotiation.cc:769] Sending connection context
0924 23:11:09.391707 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 23:11:09.391852 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260924 23:11:09.392592 19544 heartbeater.cc:347] Connected to a master server at 127.12.212.62:37579
I20260924 23:11:09.392692 19544 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:09.392871 19544 heartbeater.cc:511] Master 127.12.212.62:37579 requested a full tablet report, sending...
I20260924 23:11:09.393244 19219 ts_manager.cc:194] Registered new tserver with Master: 247c95b3bbae46d395cb9822ac4f9225 (127.12.212.2:38875)
I20260924 23:11:09.393617 19219 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:60383
WARNING: no policy specified for oryx/127.12.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.3.keytab.
Entry for principal oryx/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.3.keytab.
I20260924 23:11:09.418656 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:37579
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.3.keytab
--principal=oryx/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:09.500694 19549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:09.500979 19549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:09.501005 19549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:09.502280 19549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:09.502336 19549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:09.502363 19549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:11:09.504108 19549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.3.keytab
--principal=oryx/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:37579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:09.504369 19549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:09.504617 19549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:09.509928 19555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:09.509929 19554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:09.509928 19557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:09.510212 19549 server_base.cc:1034] running on GCE node
I20260924 23:11:09.510471 19549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:09.510742 19549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:09.511932 19549 hybrid_clock.cc:648] HybridClock initialized: now 1790291469511898 us; error 49 us; skew 500 ppm
Sep 24 23:11:09 dist-test-slave-m0z6 krb5kdc[19174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291469, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:09.514616 19549 init.cc:377] Logged in from keytab as oryx/127.12.212.3@KRBTEST.COM (short username oryx)
I20260924 23:11:09.515084 19549 webserver.cc:460] Webserver started at http://127.12.212.3:38549/ using document root <none> and password file <none>
I20260924 23:11:09.515273 19549 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:09.515312 19549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:09.515400 19549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:09.516408 19549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "0beb2bfd73a54e99a82ae77cfb709722"
format_stamp: "Formatted at 2026-09-24 23:11:09 on dist-test-slave-m0z6"
server_key: "6da8affe28750afd9b6daf990d65694c"
server_key_iv: "75bf8b41eeed64d0df1ee3bfd118dfe0"
server_key_version: "encryptionkey@0"
I20260924 23:11:09.516697 19549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "0beb2bfd73a54e99a82ae77cfb709722"
format_stamp: "Formatted at 2026-09-24 23:11:09 on dist-test-slave-m0z6"
server_key: "6da8affe28750afd9b6daf990d65694c"
server_key_iv: "75bf8b41eeed64d0df1ee3bfd118dfe0"
server_key_version: "encryptionkey@0"
I20260924 23:11:09.517995 19549 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:11:09.518858 19564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:09.519042 19549 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:09.519125 19549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "0beb2bfd73a54e99a82ae77cfb709722"
format_stamp: "Formatted at 2026-09-24 23:11:09 on dist-test-slave-m0z6"
server_key: "6da8affe28750afd9b6daf990d65694c"
server_key_iv: "75bf8b41eeed64d0df1ee3bfd118dfe0"
server_key_version: "encryptionkey@0"
I20260924 23:11:09.519184 19549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:09.552366 19549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:09.555119 19549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:09.555241 19549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:09.555508 19549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:09.555835 19549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:09.555877 19549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:09.555903 19549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:09.555927 19549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:09.561429 19549 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:36715
I20260924 23:11:09.561474 19677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:36715 every 8 connection(s)
I20260924 23:11:09.561776 19549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:11:09.564175 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 19549
I20260924 23:11:09.564263 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:11:09.564428 13136 external_mini_cluster.cc:1428] Setting key 478285d4025f20d7b14785b3274f4366
Sep 24 23:11:09 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291469, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM
I20260924 23:11:09.571884 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.562494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:39087 (local address 127.12.212.62:37579)
0924 23:11:09.562561 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:09.562563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:09.562733 (+ 170us) server_negotiation.cc:407] Connection header received
0924 23:11:09.563295 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:09.563296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:09.563306 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:09.563404 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:09.564326 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.564973 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.566501 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.566758 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.566838 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.569274 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:09.569276 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:09.569276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:09.569284 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:09.570572 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.571045 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.571047 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.571047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.571106 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.571333 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.571334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.571334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.571457 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:09.571509 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:09.571749 (+ 240us) server_negotiation.cc:299] Negotiation successful
0924 23:11:09.571816 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 23:11:09.572082 19680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.562425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.3:39087)
0924 23:11:09.562598 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:09.562634 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:09.563190 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:09.563417 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:09.563420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:09.564005 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:09.564202 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:09.564206 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.565092 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.565094 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:09.566396 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:09.566399 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.566845 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.566846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:09.566976 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.567609 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:09.567619 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:09.569160 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:09.570663 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:09.570669 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:09.570671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:09.570973 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:09.571175 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:09.571176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:09.571176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:09.571282 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:09.571509 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:09.571512 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:09.571635 (+ 123us) client_negotiation.cc:769] Sending connection context
0924 23:11:09.571717 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 23:11:09.571841 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
I20260924 23:11:09.572651 19678 heartbeater.cc:347] Connected to a master server at 127.12.212.62:37579
I20260924 23:11:09.572746 19678 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:09.572930 19678 heartbeater.cc:511] Master 127.12.212.62:37579 requested a full tablet report, sending...
I20260924 23:11:09.573392 19219 ts_manager.cc:194] Registered new tserver with Master: 0beb2bfd73a54e99a82ae77cfb709722 (127.12.212.3:36715)
I20260924 23:11:09.573774 19219 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:39087
I20260924 23:11:09.576496 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.12.212.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.212.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.212.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.61.keytab.
Entry for principal oryx/127.12.212.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.61.keytab.
I20260924 23:11:09.602977 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.61:41905
--webserver_interface=127.12.212.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.61.keytab
--principal=oryx/127.12.212.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:37579,127.12.212.61:41905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:09.685410 19683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:09.685635 19683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:09.685657 19683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:09.686924 19683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:11:09.686972 19683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:09.686985 19683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:09.686997 19683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:11:09.687007 19683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:11:09.688544 19683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:37579,127.12.212.61:41905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.61.keytab
--principal=oryx/127.12.212.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.61:41905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.212.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
W20260924 23:11:09.688649 19683 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.12.212.62:37579,127.12.212.61:41905'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260924 23:11:09.688803 19683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:09.689029 19683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:09.693786 19688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:09.693773 19691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:09.693770 19689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:09.693924 19683 server_base.cc:1034] running on GCE node
I20260924 23:11:09.694192 19683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:09.694540 19683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:09.695698 19683 hybrid_clock.cc:648] HybridClock initialized: now 1790291469695658 us; error 56 us; skew 500 ppm
Sep 24 23:11:09 dist-test-slave-m0z6 krb5kdc[19174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291469, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:09.698402 19683 init.cc:377] Logged in from keytab as oryx/127.12.212.61@KRBTEST.COM (short username oryx)
I20260924 23:11:09.698966 19683 webserver.cc:460] Webserver started at http://127.12.212.61:40639/ using document root <none> and password file <none>
I20260924 23:11:09.699189 19683 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:09.699239 19683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:09.699371 19683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:09.700424 19683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/data/instance:
uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b"
format_stamp: "Formatted at 2026-09-24 23:11:09 on dist-test-slave-m0z6"
server_key: "1cfde845395e1f1b2ca876971572a5ba"
server_key_iv: "1040370c2c7f3bc42edbaedeb85afe0a"
server_key_version: "encryptionkey@0"
I20260924 23:11:09.700759 19683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/wal/instance:
uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b"
format_stamp: "Formatted at 2026-09-24 23:11:09 on dist-test-slave-m0z6"
server_key: "1cfde845395e1f1b2ca876971572a5ba"
server_key_iv: "1040370c2c7f3bc42edbaedeb85afe0a"
server_key_version: "encryptionkey@0"
I20260924 23:11:09.702099 19683 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:11:09.703033 19698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:09.703235 19683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:09.703331 19683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/wal
uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b"
format_stamp: "Formatted at 2026-09-24 23:11:09 on dist-test-slave-m0z6"
server_key: "1cfde845395e1f1b2ca876971572a5ba"
server_key_iv: "1040370c2c7f3bc42edbaedeb85afe0a"
server_key_version: "encryptionkey@0"
I20260924 23:11:09.703400 19683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:09.715040 19683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:09.717300 19683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:09.717447 19683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:09.721252 19683 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.61:41905
I20260924 23:11:09.721294 19750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.61:41905 every 8 connection(s)
I20260924 23:11:09.721616 19683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/data/info.pb
I20260924 23:11:09.722677 19751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:09.724375 19751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:11:09.727977 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 19683
Sep 24 23:11:09 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291469, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.61@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM
Sep 24 23:11:09 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.212.61@KRBTEST.COM
I20260924 23:11:09.734586 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.725221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35038 (local address 127.12.212.62:37579)
0924 23:11:09.725284 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:09.725286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:09.725509 (+ 223us) server_negotiation.cc:407] Connection header received
0924 23:11:09.725969 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:09.725971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:09.725982 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:09.726046 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:09.726759 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.727361 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.728528 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.728814 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.728870 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.731127 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:09.731129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:09.731129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:09.731138 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:09.732719 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.733371 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.733372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.733372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.733437 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.733784 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.733786 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.733786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.733951 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:09.734014 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:09.734399 (+ 385us) server_negotiation.cc:299] Negotiation successful
0924 23:11:09.734496 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 23:11:09.734779 19753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.725110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35038)
0924 23:11:09.725311 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:09.725353 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:09.725849 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:09.726148 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:09.726152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:09.726432 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:09.726635 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:09.726639 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.727451 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.727452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:09.728403 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:09.728407 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.728895 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.728896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:09.729048 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.729522 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:09.729530 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:09.731032 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:09.732834 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:09.732843 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:09.732846 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:09.733160 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:09.733535 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:09.733535 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:09.733536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:09.733692 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:09.734036 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:09.734040 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:09.734226 (+ 186us) client_negotiation.cc:769] Sending connection context
0924 23:11:09.734335 (+ 109us) client_negotiation.cc:240] Negotiation successful
0924 23:11:09.734496 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":88,"threads_started":1}
I20260924 23:11:09.735682 19751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:11:09.737205 19755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.728470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46164 (local address 127.12.212.61:41905)
0924 23:11:09.728665 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:09.728668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:09.728758 (+ 90us) server_negotiation.cc:407] Connection header received
0924 23:11:09.728816 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:09.728818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:09.728834 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:09.728938 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:09.729471 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.730208 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.731170 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.731565 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.731636 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.733973 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:09.733981 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:09.733981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:09.733988 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:09.735778 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.736166 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.736169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.736169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.736234 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.736557 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.736558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.736559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.736738 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:09.736919 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:09.737022 (+ 103us) server_negotiation.cc:299] Negotiation successful
0924 23:11:09.737105 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":97,"threads_started":1}
Sep 24 23:11:09 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291469, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.61@KRBTEST.COM for oryx/127.12.212.61@KRBTEST.COM
I20260924 23:11:09.743752 19753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.735993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:41905 (local address 127.0.0.1:46172)
0924 23:11:09.736046 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:09.736056 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:09.736089 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:09.736293 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:09.736294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:09.736430 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:09.736497 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:09.736499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.737671 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.737673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:09.738316 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:09.738318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.738775 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.738776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:09.738843 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.739292 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:09.739297 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:09.740575 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:09.742761 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:09.742762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:09.742762 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:09.742930 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:09.743119 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:09.743119 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:09.743120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:09.743169 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:09.743421 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:09.743421 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:09.743461 (+ 40us) client_negotiation.cc:769] Sending connection context
0924 23:11:09.743522 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 23:11:09.743633 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 23:11:09.743795 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.736028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46172 (local address 127.12.212.61:41905)
0924 23:11:09.736178 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:09.736180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:09.736186 (+ 6us) server_negotiation.cc:407] Connection header received
0924 23:11:09.736199 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:09.736200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:09.736208 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:09.736274 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:09.736586 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.737545 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.738388 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.738606 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.738663 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.740676 (+ 2013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:09.740678 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:09.740678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:09.740686 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:09.742633 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.743002 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.743002 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.743003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.743058 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.743239 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.743240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.743240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.743362 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:09.743428 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:09.743555 (+ 127us) server_negotiation.cc:299] Negotiation successful
0924 23:11:09.743685 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260924 23:11:09.745206 19751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: Bootstrap starting.
I20260924 23:11:09.745868 19751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:09.746132 19751 log.cc:824] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:09.747021 19751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: No bootstrap required, opened a new log
I20260924 23:11:09.748495 19751 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } }
I20260924 23:11:09.748629 19751 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:09.748651 19751 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfc0464cc67a40de8e97ffd2dfb0c48b, State: Initialized, Role: FOLLOWER
I20260924 23:11:09.748768 19751 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } }
I20260924 23:11:09.749084 19751 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:11:09.749074 19758 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } } }
I20260924 23:11:09.749164 19758 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [sys.catalog]: This master's current role is: FOLLOWER
W20260924 23:11:09.750672 19769 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:11:09.750762 19769 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:11:09.752640 19683 master_runner.cc:428] Detected that this master cfc0464cc67a40de8e97ffd2dfb0c48b is joining an existing cluster
I20260924 23:11:09.752691 19683 master_runner.cc:432] Initiating AddMaster RPC to add 127.12.212.61:41905
I20260924 23:11:09.760191 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.753981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46184 (local address 127.12.212.61:41905)
0924 23:11:09.754021 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:09.754022 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:09.754077 (+ 55us) server_negotiation.cc:407] Connection header received
0924 23:11:09.754128 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:09.754129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:09.754136 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:09.754184 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:09.754576 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.755324 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.756023 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.756246 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.756301 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.757509 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:09.757510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:09.757510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:09.757516 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:09.759054 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.759362 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.759363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.759364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.759412 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.759577 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.759578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.759579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.759696 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:09.759760 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:09.759961 (+ 201us) server_negotiation.cc:299] Negotiation successful
0924 23:11:09.760083 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":22}
I20260924 23:11:09.760643 19776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.753913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:41905 (local address 127.0.0.1:46184)
0924 23:11:09.754033 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:09.754041 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:09.754087 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:09.754190 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:09.754191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:09.754439 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:09.754502 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:09.754504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.755395 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.755395 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:09.755945 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:09.755946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.756334 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.756336 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:09.756388 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.756952 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:09.756955 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:09.757422 (+ 467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:09.759128 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:09.759129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:09.759130 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:09.759297 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:09.759468 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:09.759469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:09.759469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:09.759522 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:09.759756 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:09.759757 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:09.759797 (+ 40us) client_negotiation.cc:769] Sending connection context
0924 23:11:09.759897 (+ 100us) client_negotiation.cc:240] Negotiation successful
0924 23:11:09.760012 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":119,"threads_started":1}
I20260924 23:11:09.760934 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.753722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35042 (local address 127.12.212.62:37579)
0924 23:11:09.753765 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:09.753767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:09.753949 (+ 182us) server_negotiation.cc:407] Connection header received
0924 23:11:09.753977 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:09.753978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:09.753990 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:09.754044 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:09.754477 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.755295 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.756392 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.756667 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.756724 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.757582 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:09.757584 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:09.757584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:09.757593 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:09.759502 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.760127 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.760129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.760129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.760187 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.760383 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.760384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.760384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.760511 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:09.760550 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:09.760768 (+ 218us) server_negotiation.cc:299] Negotiation successful
0924 23:11:09.760855 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 23:11:09.761209 19775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.753652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35042)
0924 23:11:09.753824 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:09.753837 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:09.753908 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:09.754091 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:09.754092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:09.754325 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:09.754387 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:09.754389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.755383 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.755384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:09.756300 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:09.756302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.756744 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.756745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:09.756807 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.757109 (+ 302us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:09.757113 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:09.757488 (+ 375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:09.759781 (+ 2293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:09.759782 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:09.759782 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:09.760042 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:09.760249 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:09.760250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:09.760250 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:09.760310 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:09.760591 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:09.760592 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:09.760630 (+ 38us) client_negotiation.cc:769] Sending connection context
0924 23:11:09.761006 (+ 376us) client_negotiation.cc:240] Negotiation successful
0924 23:11:09.761114 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":125,"threads_started":1}
Sep 24 23:11:09 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291468, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.62@KRBTEST.COM for oryx/127.12.212.61@KRBTEST.COM
I20260924 23:11:09.771522 19777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.763582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:41905 (local address 127.0.0.1:46200)
0924 23:11:09.763708 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:09.763751 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:09.763826 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:09.763962 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:09.763964 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:09.764218 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:09.764278 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:09.764279 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.765175 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.765176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:09.765820 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:09.765822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.766326 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.766327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:09.766393 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.766910 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:09.766914 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:09.768556 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:09.770328 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:09.770335 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:09.770337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:09.770555 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:09.770783 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:09.770784 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:09.770784 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:09.770855 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:09.771166 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:09.771174 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:09.771228 (+ 54us) client_negotiation.cc:769] Sending connection context
0924 23:11:09.771310 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 23:11:09.771435 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":77,"threads_started":1}
I20260924 23:11:09.771539 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.763671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46200 (local address 127.12.212.61:41905)
0924 23:11:09.763709 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:09.763712 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:09.763821 (+ 109us) server_negotiation.cc:407] Connection header received
0924 23:11:09.763882 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:09.763883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:09.763893 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:09.763958 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:09.764346 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.765081 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.765903 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.766202 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.766268 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.768644 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:09.768645 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:09.768646 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:09.768653 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:09.770191 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.770632 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.770633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.770633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.770719 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.770929 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.770929 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.770930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.771073 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:09.771141 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:09.771365 (+ 224us) server_negotiation.cc:299] Negotiation successful
0924 23:11:09.771445 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 23:11:09.772178 19219 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.12.212.61:41905: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b"
member_type: NON_VOTER
last_known_addr {
host: "127.12.212.61"
port: 41905
}
attrs {
promote: true
}
}
dest_uuid: "838e7664b2b344b980d6324a8a2e3ec4"
cas_config_opid_index: -1
I20260924 23:11:09.772531 19219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: NON_VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: true } }
I20260924 23:11:09.773032 19779 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER cfc0464cc67a40de8e97ffd2dfb0c48b (127.12.212.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: NON_VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: true } } }
I20260924 23:11:09.773432 19725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:09.774049 19779 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.12.212.61:41905
I20260924 23:11:09.774196 19725 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 1 FOLLOWER]: Refusing update from remote peer 838e7664b2b344b980d6324a8a2e3ec4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260924 23:11:09.774274 19683 master.cc:537] Master@127.12.212.61:41905 shutting down...
I20260924 23:11:09.774523 19779 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: NON_VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:09.775009 19778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "838e7664b2b344b980d6324a8a2e3ec4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: NON_VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: true } } }
I20260924 23:11:09.775086 19778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: This master's current role is: LEADER
W20260924 23:11:09.775291 19204 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260924 23:11:09.775692 19204 consensus_peers.cc:487] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 -> Peer cfc0464cc67a40de8e97ffd2dfb0c48b (127.12.212.61:41905): Couldn't send request to peer cfc0464cc67a40de8e97ffd2dfb0c48b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260924 23:11:09.777371 19683 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:11:09.777464 19683 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:09.777513 19683 tablet_replica.cc:330] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: stopping tablet replica
I20260924 23:11:09.790000 19683 master.cc:552] Master@127.12.212.61:41905 shutdown complete.
I20260924 23:11:09.790056 19683 master_runner.cc:305] Clearing existing system tablet
I20260924 23:11:09.790181 19683 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:09.790225 19683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:09.790838 19683 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:09.791307 19784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:09.791419 19683 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:09.791472 19683 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:09.791495 19683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/wal
uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b"
format_stamp: "Formatted at 2026-09-24 23:11:09 on dist-test-slave-m0z6"
server_key: "1cfde845395e1f1b2ca876971572a5ba"
server_key_iv: "1040370c2c7f3bc42edbaedeb85afe0a"
server_key_version: "encryptionkey@0"
I20260924 23:11:09.792189 19683 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:09.793987 19683 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260924 23:11:09.794060 19683 log.cc:1196] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260924 23:11:09.794256 19683 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: Deleting consensus metadata
I20260924 23:11:09.794680 19683 master_runner.cc:315] Copying system tablet from 127.12.212.62:37579
I20260924 23:11:09.795506 19683 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: tablet copy: Beginning tablet copy session from remote peer at address 127.12.212.62:37579
I20260924 23:11:09.802305 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.795919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35058 (local address 127.12.212.62:37579)
0924 23:11:09.795962 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:09.795964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:09.796118 (+ 154us) server_negotiation.cc:407] Connection header received
0924 23:11:09.796189 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:09.796190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:09.796201 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:09.796258 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:09.796644 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.797402 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.798224 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.798427 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.798485 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.799584 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:09.799585 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:09.799586 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:09.799595 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:09.801091 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.801507 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.801508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.801508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.801568 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.801771 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.801773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.801773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.801896 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:09.801958 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:09.802112 (+ 154us) server_negotiation.cc:299] Negotiation successful
0924 23:11:09.802194 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 23:11:09.802317 19790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.795859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35058)
0924 23:11:09.796016 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:09.796026 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:09.796123 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:09.796271 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:09.796272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:09.796483 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:09.796544 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:09.796546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.797535 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.797536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:09.798126 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:09.798128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.798506 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.798507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:09.798577 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.799135 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:09.799140 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:09.799459 (+ 319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:09.801209 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:09.801210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:09.801211 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:09.801418 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:09.801636 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:09.801638 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:09.801638 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:09.801693 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:09.801965 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:09.801966 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:09.802015 (+ 49us) client_negotiation.cc:769] Sending connection context
0924 23:11:09.802073 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 23:11:09.802195 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":76,"threads_started":1}
I20260924 23:11:09.802973 19239 tablet_copy_service.cc:140] P 838e7664b2b344b980d6324a8a2e3ec4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cfc0464cc67a40de8e97ffd2dfb0c48b ({username='oryx', principal='oryx/127.12.212.61@KRBTEST.COM'} at 127.0.0.1:35058)
I20260924 23:11:09.803117 19239 tablet_copy_service.cc:161] P 838e7664b2b344b980d6324a8a2e3ec4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cfc0464cc67a40de8e97ffd2dfb0c48b at {username='oryx', principal='oryx/127.12.212.61@KRBTEST.COM'} at 127.0.0.1:35058: session id = cfc0464cc67a40de8e97ffd2dfb0c48b-00000000000000000000000000000000
I20260924 23:11:09.804011 19239 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 23:11:09.804831 19683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260924 23:11:09.805851 19683 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: tablet copy: Starting download of 0 data blocks...
I20260924 23:11:09.805998 19683 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: tablet copy: Starting download of 1 WAL segments...
I20260924 23:11:09.807068 19683 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 23:11:09.807973 19239 tablet_copy_service.cc:342] P 838e7664b2b344b980d6324a8a2e3ec4: Request end of tablet copy session cfc0464cc67a40de8e97ffd2dfb0c48b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.12.212.61@KRBTEST.COM'} at 127.0.0.1:35058
I20260924 23:11:09.808032 19239 tablet_copy_service.cc:433] P 838e7664b2b344b980d6324a8a2e3ec4: ending tablet copy session cfc0464cc67a40de8e97ffd2dfb0c48b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cfc0464cc67a40de8e97ffd2dfb0c48b
I20260924 23:11:09.810528 19683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:09.811878 19792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:09.812013 19683 server_base.cc:1034] running on GCE node
W20260924 23:11:09.811925 19791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:09.814841 19794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:09.815018 19683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:09.815186 19683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:09.816326 19683 hybrid_clock.cc:648] HybridClock initialized: now 1790291469816319 us; error 28 us; skew 500 ppm
Sep 24 23:11:09 dist-test-slave-m0z6 krb5kdc[19174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291469, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:09.817794 19683 init.cc:377] Logged in from keytab as oryx/127.12.212.61@KRBTEST.COM (short username oryx)
I20260924 23:11:09.818073 19683 webserver.cc:460] Webserver started at http://127.12.212.61:36143/ using document root <none> and password file <none>
I20260924 23:11:09.818190 19683 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:09.818221 19683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:09.818863 19683 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260924 23:11:09.819514 19801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:09.819666 19683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:11:09.819730 19683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/wal
uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b"
format_stamp: "Formatted at 2026-09-24 23:11:09 on dist-test-slave-m0z6"
server_key: "1cfde845395e1f1b2ca876971572a5ba"
server_key_iv: "1040370c2c7f3bc42edbaedeb85afe0a"
server_key_version: "encryptionkey@0"
I20260924 23:11:09.819794 19683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:09.863914 19683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:09.865818 19683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:09.869051 19683 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.61:41905
I20260924 23:11:09.869088 19853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.61:41905 every 8 connection(s)
I20260924 23:11:09.869413 19683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-1/data/info.pb
I20260924 23:11:09.870220 19854 sys_catalog.cc:263] Verifying existing consensus state
I20260924 23:11:09.870641 19854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: Bootstrap starting.
I20260924 23:11:09.873051 19854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 23:11:09.873270 19854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: Bootstrap complete.
I20260924 23:11:09.873478 19854 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: NON_VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: true } }
I20260924 23:11:09.873548 19854 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cfc0464cc67a40de8e97ffd2dfb0c48b, State: Initialized, Role: LEARNER
I20260924 23:11:09.873584 19854 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: NON_VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: true } }
I20260924 23:11:09.873751 19854 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:11:09.873755 19857 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: NON_VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: true } } }
I20260924 23:11:09.873891 19857 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [sys.catalog]: This master's current role is: LEARNER
I20260924 23:11:09.875161 19868 catalog_manager.cc:1259] Loaded cluster ID: 95821273abe2409e9ab3091d80765996
I20260924 23:11:09.875233 19868 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: loading cluster ID for follower catalog manager: success
I20260924 23:11:09.876140 19868 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: acquiring CA information for follower catalog manager: success
I20260924 23:11:09.876406 19868 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 23:11:09.896486 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:09.889653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46230 (local address 127.12.212.61:41905)
0924 23:11:09.889829 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:09.889831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:09.889838 (+ 7us) server_negotiation.cc:407] Connection header received
0924 23:11:09.889861 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:09.889862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:09.889872 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:09.889941 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:09.890604 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.891428 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.892151 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:09.892414 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:09.892477 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:09.893764 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:09.893765 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:09.893765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:09.893773 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:09.895352 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.895695 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.895697 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.895697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.895749 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:09.895930 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:09.895931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:09.895932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:09.896043 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:09.896134 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:09.896240 (+ 106us) server_negotiation.cc:299] Negotiation successful
0924 23:11:09.896385 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for oryx/127.12.212.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.212.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.212.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.60.keytab.
Entry for principal oryx/127.12.212.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.60.keytab.
I20260924 23:11:09.921948 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.60:45965
--webserver_interface=127.12.212.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.60.keytab
--principal=oryx/127.12.212.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:37579,127.12.212.61:41905,127.12.212.60:45965
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:10.006850 19872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:10.007076 19872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:10.007107 19872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:10.008378 19872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:11:10.008428 19872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:10.008447 19872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:10.008464 19872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:11:10.008479 19872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:11:10.010178 19872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:37579,127.12.212.61:41905,127.12.212.60:45965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.60.keytab
--principal=oryx/127.12.212.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.60:45965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.212.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:10.010459 19872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:10.010680 19872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:11:10.016009 19872 server_base.cc:1034] running on GCE node
W20260924 23:11:10.015961 19877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:10.015961 19878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:10.015967 19880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:10.016475 19872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:10.016799 19872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:10.017974 19872 hybrid_clock.cc:648] HybridClock initialized: now 1790291470017958 us; error 29 us; skew 500 ppm
Sep 24 23:11:10 dist-test-slave-m0z6 krb5kdc[19174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291470, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:10.020714 19872 init.cc:377] Logged in from keytab as oryx/127.12.212.60@KRBTEST.COM (short username oryx)
I20260924 23:11:10.021242 19872 webserver.cc:460] Webserver started at http://127.12.212.60:36889/ using document root <none> and password file <none>
I20260924 23:11:10.021477 19872 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:10.021529 19872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:10.021669 19872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:10.022857 19872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/data/instance:
uuid: "624c4e6f108b4c828263befbc5ae6c5d"
format_stamp: "Formatted at 2026-09-24 23:11:10 on dist-test-slave-m0z6"
server_key: "2388f5238bec870f5eda7658789049ed"
server_key_iv: "8c7f01f382cdd818b9af9cb73f90b273"
server_key_version: "encryptionkey@0"
I20260924 23:11:10.023277 19872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/wal/instance:
uuid: "624c4e6f108b4c828263befbc5ae6c5d"
format_stamp: "Formatted at 2026-09-24 23:11:10 on dist-test-slave-m0z6"
server_key: "2388f5238bec870f5eda7658789049ed"
server_key_iv: "8c7f01f382cdd818b9af9cb73f90b273"
server_key_version: "encryptionkey@0"
I20260924 23:11:10.024703 19872 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:11:10.025506 19887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:10.025722 19872 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:10.025810 19872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/wal
uuid: "624c4e6f108b4c828263befbc5ae6c5d"
format_stamp: "Formatted at 2026-09-24 23:11:10 on dist-test-slave-m0z6"
server_key: "2388f5238bec870f5eda7658789049ed"
server_key_iv: "8c7f01f382cdd818b9af9cb73f90b273"
server_key_version: "encryptionkey@0"
I20260924 23:11:10.025902 19872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:10.037694 19872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:10.040652 19872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:10.040808 19872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:10.045086 19872 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.60:45965
I20260924 23:11:10.045118 19939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.60:45965 every 8 connection(s)
I20260924 23:11:10.045493 19872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/data/info.pb
I20260924 23:11:10.046464 19940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:10.047228 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 19872
I20260924 23:11:10.048405 19940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 23:11:10 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291470, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.60@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM
Sep 24 23:11:10 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.212.60@KRBTEST.COM
I20260924 23:11:10.059814 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.049183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35074 (local address 127.12.212.62:37579)
0924 23:11:10.049272 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.049273 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.049348 (+ 75us) server_negotiation.cc:407] Connection header received
0924 23:11:10.049395 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.049396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.049407 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.049460 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.050245 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.051100 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.052935 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.053321 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.053557 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.056240 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.056243 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.056243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.056252 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.058075 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.058600 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.058601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.058602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.058666 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.059022 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.059024 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.059024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.059174 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.059235 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.059562 (+ 327us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.059685 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260924 23:11:10.059958 19943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.049057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35074)
0924 23:11:10.049223 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.049264 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.049339 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.049502 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.049505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.049841 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:10.050095 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.050102 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.051243 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.051244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.052802 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.052805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.053579 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.053580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.053644 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.054334 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:10.054346 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:10.056123 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:10.058218 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.058224 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.058227 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.058497 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.058759 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.058760 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.058761 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.058923 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.059256 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:10.059263 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:10.059415 (+ 152us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.059508 (+ 93us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.059695 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260924 23:11:10.060063 19942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.048096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52262 (local address 127.12.212.60:45965)
0924 23:11:10.048295 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.048334 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.048363 (+ 29us) server_negotiation.cc:407] Connection header received
0924 23:11:10.049095 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.049106 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.049190 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.049341 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.050246 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.051439 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.052829 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.053468 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.053594 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.056719 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.056728 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.056729 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.056748 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.058366 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.058918 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.058921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.058922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.058995 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.059238 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.059239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.059239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.059412 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.059535 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.059884 (+ 349us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.059958 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":84,"threads_started":1}
I20260924 23:11:10.061215 19940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 23:11:10 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291470, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.60@KRBTEST.COM for oryx/127.12.212.61@KRBTEST.COM
I20260924 23:11:10.069171 19943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.061531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:41905 (local address 127.0.0.1:46236)
0924 23:11:10.061587 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.061599 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.061641 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.061835 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.061836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.061975 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:10.062024 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.062026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.063079 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.063081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.063648 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.063650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.064251 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.064252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.064329 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.064962 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:10.064966 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:10.066161 (+ 1195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:10.067957 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.067958 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.067958 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.068141 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.068473 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.068473 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.068474 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.068536 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.068870 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:10.068871 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:10.068925 (+ 54us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.069002 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.069096 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 23:11:10.069233 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.061647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46236 (local address 127.12.212.61:41905)
0924 23:11:10.061702 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.061704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.061713 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:11:10.061735 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.061735 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.061748 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.061793 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.062132 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.062960 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.063773 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.064094 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.064167 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.066273 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.066275 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.066276 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.066284 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.067859 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.068239 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.068240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.068241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.068366 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.068644 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.068645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.068646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.068768 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.068823 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.069069 (+ 246us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.069160 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 23:11:10.069692 19940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 23:11:10 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291470, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.60@KRBTEST.COM for oryx/127.12.212.60@KRBTEST.COM
I20260924 23:11:10.076709 19942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.070114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52272 (local address 127.12.212.60:45965)
0924 23:11:10.070170 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.070172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.070178 (+ 6us) server_negotiation.cc:407] Connection header received
0924 23:11:10.070194 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.070195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.070203 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.070245 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.070509 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.071049 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.071677 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.071885 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.071940 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.073920 (+ 1980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.073922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.073922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.073929 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.075544 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.075979 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.075981 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.075981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.076047 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.076199 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.076199 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.076200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.076318 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.076375 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.076508 (+ 133us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.076593 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":40}
I20260924 23:11:10.076720 19943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.070028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:45965 (local address 127.0.0.1:52272)
0924 23:11:10.070084 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.070094 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.070132 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.070263 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.070264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.070403 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:10.070439 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.070441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.071120 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.071121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.071600 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.071601 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.071951 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.071952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.071999 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.072474 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:10.072479 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:10.073801 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:10.075632 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.075633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.075634 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.075873 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.076109 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.076110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.076110 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.076155 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.076378 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:10.076379 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:10.076426 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.076480 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.076636 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":65}
I20260924 23:11:10.078326 19940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: Bootstrap starting.
I20260924 23:11:10.079558 19940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:10.079918 19940 log.cc:824] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:10.080790 19940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: No bootstrap required, opened a new log
I20260924 23:11:10.082525 19940 raft_consensus.cc:348] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } }
I20260924 23:11:10.082686 19940 raft_consensus.cc:374] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:10.082762 19940 raft_consensus.cc:729] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 624c4e6f108b4c828263befbc5ae6c5d, State: Initialized, Role: FOLLOWER
I20260924 23:11:10.082921 19940 consensus_queue.cc:260] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } }
I20260924 23:11:10.083261 19945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } } }
I20260924 23:11:10.083331 19940 sys_catalog.cc:564] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:11:10.083346 19945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [sys.catalog]: This master's current role is: FOLLOWER
W20260924 23:11:10.085021 19956 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:11:10.085084 19956 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:11:10.087929 19872 master_runner.cc:428] Detected that this master 624c4e6f108b4c828263befbc5ae6c5d is joining an existing cluster
I20260924 23:11:10.087990 19872 master_runner.cc:432] Initiating AddMaster RPC to add 127.12.212.60:45965
I20260924 23:11:10.111433 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 19281
I20260924 23:11:10.116056 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.089316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46250 (local address 127.12.212.61:41905)
0924 23:11:10.089355 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.089357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.089449 (+ 92us) server_negotiation.cc:407] Connection header received
0924 23:11:10.089532 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.089533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.089544 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.089572 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.090002 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.090769 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.092013 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.092316 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.092352 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.111908 (+ 19556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.111911 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.111911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.111920 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.113783 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.115206 (+ 1423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.115208 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.115208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.115283 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.115480 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.115481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.115481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.115622 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.115662 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.115859 (+ 197us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.115954 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 23:11:10.116385 19963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.089171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:41905 (local address 127.0.0.1:46250)
0924 23:11:10.089400 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.089409 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.089468 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.089631 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.089632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.089858 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:10.089922 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.089925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.090846 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.090846 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.091874 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.091876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.092427 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.092428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.092495 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.093102 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:10.093107 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:10.111789 (+ 18682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:10.114602 (+ 2813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.114602 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.114603 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.114836 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.115357 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.115357 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.115357 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.115415 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.115705 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:10.115705 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:10.115752 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.116152 (+ 400us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.116284 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":89,"thread_start_us":54,"threads_started":1}
I20260924 23:11:10.116981 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.089047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35084 (local address 127.12.212.62:37579)
0924 23:11:10.089130 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.089132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.089174 (+ 42us) server_negotiation.cc:407] Connection header received
0924 23:11:10.089280 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.089281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.089294 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.089326 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.089881 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.090727 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.091983 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.092287 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.092328 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.113089 (+ 20761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.113090 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.113091 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.113270 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.114909 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.115392 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.115394 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.115394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.115460 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.115924 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.115925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.115925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.116065 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.116106 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.116801 (+ 695us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.116889 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 23:11:10.117024 19942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.089090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52286 (local address 127.12.212.60:45965)
0924 23:11:10.089160 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.089161 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.089499 (+ 338us) server_negotiation.cc:407] Connection header received
0924 23:11:10.089594 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.089595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.089603 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.089628 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.090062 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.090772 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.091875 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.092156 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.092195 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.112312 (+ 20117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.112314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.112314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.112321 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.114530 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.115724 (+ 1194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.115725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.115725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.115764 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.116526 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.116527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.116528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.116661 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.116700 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.116864 (+ 164us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.116937 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 23:11:10.117276 19964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.089213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:45965 (local address 127.0.0.1:52286)
0924 23:11:10.089440 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.089449 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.089526 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.089682 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.089682 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.089914 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:10.089979 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.089981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.090858 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.090859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.091776 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.091779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.092296 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.092297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.092354 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.092801 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:10.092806 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:10.111347 (+ 18541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:10.114942 (+ 3595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.114943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.114943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.115528 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.115816 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.115816 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.115817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.115859 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.116738 (+ 879us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:10.116738 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:10.116773 (+ 35us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.117086 (+ 313us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.117169 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":160,"threads_started":1}
I20260924 23:11:10.117625 19962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.088977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35084)
0924 23:11:10.089110 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.089122 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.089202 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.089398 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.089398 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.089696 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:10.089789 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.089791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.090824 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.090826 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.091874 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.091876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.092426 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.092427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.092495 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.111291 (+ 18796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:10.111297 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:10.112804 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:10.115012 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.115012 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.115013 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.115294 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.115595 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.115595 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.115596 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.115664 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.116618 (+ 954us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:10.116619 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:10.116681 (+ 62us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.117426 (+ 745us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.117528 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":71,"thread_start_us":78,"threads_started":1}
I20260924 23:11:10.118768 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:33293
--local_ip_for_outbound_sockets=127.12.212.1
--tserver_master_addrs=127.12.212.62:37579
--webserver_port=36737
--webserver_interface=127.12.212.1
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.1.keytab
--principal=oryx/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.212.62:37579,127.12.212.61:41905
--tserver_master_addrs=127.12.212.62:37579,127.12.212.61:41905,127.12.212.60:45965 with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 23:11:10 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291468, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.62@KRBTEST.COM for oryx/127.12.212.60@KRBTEST.COM
I20260924 23:11:10.128203 19942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.120397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52294 (local address 127.12.212.60:45965)
0924 23:11:10.120438 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.120440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.120449 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:11:10.120484 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.120485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.120495 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.120554 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.120893 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.121551 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.122556 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.122871 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.122948 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.125166 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.125167 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.125168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.125177 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.126894 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.127328 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.127330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.127330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.127407 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.127614 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.127615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.127616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.127771 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.127845 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.128021 (+ 176us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.128106 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 23:11:10.128206 19777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.120242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:45965 (local address 127.0.0.1:52294)
0924 23:11:10.120330 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.120340 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.120376 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.120578 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.120579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.120748 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:10.120801 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.120803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.121620 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.121621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.122464 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.122467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.122955 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.122956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.123033 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.123527 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:10.123532 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:10.125059 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:10.126997 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.126998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.126999 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.127242 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.127480 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.127481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.127482 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.127547 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.127847 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:10.127848 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:10.127902 (+ 54us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.127972 (+ 70us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.128095 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260924 23:11:10.129215 19219 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.12.212.60:45965: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d"
member_type: NON_VOTER
last_known_addr {
host: "127.12.212.60"
port: 45965
}
attrs {
promote: true
}
}
dest_uuid: "838e7664b2b344b980d6324a8a2e3ec4"
cas_config_opid_index: 5
I20260924 23:11:10.129487 19219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: NON_VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: true } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: NON_VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: true } }
I20260924 23:11:10.129921 19778 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 624c4e6f108b4c828263befbc5ae6c5d (127.12.212.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: NON_VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: true } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: NON_VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: true } } }
I20260924 23:11:10.130983 19778 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.12.212.60:45965
I20260924 23:11:10.131062 19778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "838e7664b2b344b980d6324a8a2e3ec4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: NON_VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: true } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: NON_VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: true } } }
I20260924 23:11:10.131132 19778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:10.131767 19872 master.cc:537] Master@127.12.212.60:45965 shutting down...
I20260924 23:11:10.131907 19912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:10.133517 19912 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 1 FOLLOWER]: Refusing update from remote peer 838e7664b2b344b980d6324a8a2e3ec4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260924 23:11:10.134470 19780 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: NON_VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 23:11:10.135473 19205 consensus_peers.cc:487] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 -> Peer 624c4e6f108b4c828263befbc5ae6c5d (127.12.212.60:45965): Couldn't send request to peer 624c4e6f108b4c828263befbc5ae6c5d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260924 23:11:10.136811 19872 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:11:10.136926 19872 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:10.136977 19872 tablet_replica.cc:330] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: stopping tablet replica
I20260924 23:11:10.139709 19777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.131388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:41905 (local address 127.0.0.1:46262)
0924 23:11:10.135050 (+ 3662us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.135066 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.135170 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.135413 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.135413 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.135415 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:10.135465 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.135467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.136643 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.136644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.138251 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.138257 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.139402 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.139403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.139481 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.139521 (+ 40us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.139576 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.139623 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3636}
I20260924 23:11:10.139745 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.131473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46262 (local address 127.12.212.61:41905)
0924 23:11:10.131521 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.131523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.135136 (+ 3613us) server_negotiation.cc:407] Connection header received
0924 23:11:10.135246 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.135247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.135270 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.135312 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:10.135643 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.136407 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.138398 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.139063 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.139116 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.139200 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.139620 (+ 420us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.139673 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 23:11:10.140372 19828 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 1 LEARNER]: Refusing update from remote peer 838e7664b2b344b980d6324a8a2e3ec4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260924 23:11:10.140604 19780 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: NON_VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 23:11:10.141232 19828 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 624c4e6f108b4c828263befbc5ae6c5d (127.12.212.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: NON_VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: true } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: NON_VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: true } } }
I20260924 23:11:10.142462 19857 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 838e7664b2b344b980d6324a8a2e3ec4. Latest consensus state: current_term: 1 leader_uuid: "838e7664b2b344b980d6324a8a2e3ec4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: NON_VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: true } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: NON_VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: true } } }
I20260924 23:11:10.142557 19857 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [sys.catalog]: This master's current role is: LEARNER
I20260924 23:11:10.142588 19857 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "838e7664b2b344b980d6324a8a2e3ec4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: NON_VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: true } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: NON_VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: true } } }
I20260924 23:11:10.142632 19857 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [sys.catalog]: This master's current role is: LEARNER
I20260924 23:11:10.142980 19780 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4: attempting to promote NON_VOTER cfc0464cc67a40de8e97ffd2dfb0c48b to VOTER
I20260924 23:11:10.143256 19780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: NON_VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: true } }
I20260924 23:11:10.143959 19828 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 1 LEARNER]: Refusing update from remote peer 838e7664b2b344b980d6324a8a2e3ec4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260924 23:11:10.144022 19777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.143856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:45965 (local address 127.0.0.1:52300)
0924 23:11:10.143920 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.143984 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:45965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260924 23:11:10.144140 19778 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 23:11:10.144569 19777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.144485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:45965 (local address 127.0.0.1:52314)
0924 23:11:10.144523 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.144545 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:45965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 23:11:10.144634 19205 consensus_peers.cc:487] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 -> Peer 624c4e6f108b4c828263befbc5ae6c5d (127.12.212.60:45965): Couldn't send request to peer 624c4e6f108b4c828263befbc5ae6c5d. Status: Network error: Client connection negotiation failed: client connection to 127.12.212.60:45965: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 23:11:10.144975 19778 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cfc0464cc67a40de8e97ffd2dfb0c48b (127.12.212.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: NON_VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: true } } }
I20260924 23:11:10.145921 19778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "838e7664b2b344b980d6324a8a2e3ec4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: NON_VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: true } } }
I20260924 23:11:10.145992 19778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:10.146047 19778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "838e7664b2b344b980d6324a8a2e3ec4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: NON_VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: true } } }
I20260924 23:11:10.146085 19778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:10.146433 19828 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cfc0464cc67a40de8e97ffd2dfb0c48b (127.12.212.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: NON_VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: true } } }
I20260924 23:11:10.147722 19967 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "838e7664b2b344b980d6324a8a2e3ec4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: NON_VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: true } } }
I20260924 23:11:10.147861 19967 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:11:10.150230 19872 master.cc:552] Master@127.12.212.60:45965 shutdown complete.
I20260924 23:11:10.150291 19872 master_runner.cc:305] Clearing existing system tablet
I20260924 23:11:10.150417 19872 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:10.150465 19872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:10.151410 19872 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260924 23:11:10.151984 19969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:10.152135 19872 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:10.152201 19872 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:10.152232 19872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/wal
uuid: "624c4e6f108b4c828263befbc5ae6c5d"
format_stamp: "Formatted at 2026-09-24 23:11:10 on dist-test-slave-m0z6"
server_key: "2388f5238bec870f5eda7658789049ed"
server_key_iv: "8c7f01f382cdd818b9af9cb73f90b273"
server_key_version: "encryptionkey@0"
I20260924 23:11:10.152974 19872 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:10.154735 19872 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260924 23:11:10.154816 19872 log.cc:1196] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260924 23:11:10.155031 19872 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: Deleting consensus metadata
I20260924 23:11:10.155155 19872 master_runner.cc:315] Copying system tablet from 127.12.212.62:37579
I20260924 23:11:10.155783 19872 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: tablet copy: Beginning tablet copy session from remote peer at address 127.12.212.62:37579
I20260924 23:11:10.166807 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.159073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35094 (local address 127.12.212.62:37579)
0924 23:11:10.159131 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.159133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.159322 (+ 189us) server_negotiation.cc:407] Connection header received
0924 23:11:10.159374 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.159375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.159389 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.159453 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.159891 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.160653 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.161761 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.162065 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.162139 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.163505 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.163507 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.163507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.163517 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.165292 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.165816 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.165818 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.165819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.165905 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.166143 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.166144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.166144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.166297 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.166370 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.166568 (+ 198us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.166661 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 23:11:10.166836 19975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.159001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35094)
0924 23:11:10.159195 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.159208 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.159310 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.159493 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.159494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.159729 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:10.159795 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.159799 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.160767 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.160768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.161662 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.161666 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.162161 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.162162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.162235 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.162792 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:10.162797 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:10.163373 (+ 576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:10.165470 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.165472 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.165472 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.165717 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.165993 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.165994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.165994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.166072 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.166384 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:10.166385 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:10.166445 (+ 60us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.166523 (+ 78us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.166661 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":85,"threads_started":1}
I20260924 23:11:10.167557 19239 tablet_copy_service.cc:140] P 838e7664b2b344b980d6324a8a2e3ec4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 624c4e6f108b4c828263befbc5ae6c5d ({username='oryx', principal='oryx/127.12.212.60@KRBTEST.COM'} at 127.0.0.1:35094)
I20260924 23:11:10.167627 19239 tablet_copy_service.cc:161] P 838e7664b2b344b980d6324a8a2e3ec4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 624c4e6f108b4c828263befbc5ae6c5d at {username='oryx', principal='oryx/127.12.212.60@KRBTEST.COM'} at 127.0.0.1:35094: session id = 624c4e6f108b4c828263befbc5ae6c5d-00000000000000000000000000000000
I20260924 23:11:10.168236 19239 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 23:11:10.168788 19872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260924 23:11:10.170269 19872 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: tablet copy: Starting download of 0 data blocks...
I20260924 23:11:10.170418 19872 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: tablet copy: Starting download of 1 WAL segments...
I20260924 23:11:10.171397 19872 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 23:11:10.172541 19239 tablet_copy_service.cc:342] P 838e7664b2b344b980d6324a8a2e3ec4: Request end of tablet copy session 624c4e6f108b4c828263befbc5ae6c5d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.12.212.60@KRBTEST.COM'} at 127.0.0.1:35094
I20260924 23:11:10.172607 19239 tablet_copy_service.cc:433] P 838e7664b2b344b980d6324a8a2e3ec4: ending tablet copy session 624c4e6f108b4c828263befbc5ae6c5d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 624c4e6f108b4c828263befbc5ae6c5d
I20260924 23:11:10.176024 19872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:10.177455 19979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:10.177560 19976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:10.177876 19872 server_base.cc:1034] running on GCE node
W20260924 23:11:10.179625 19977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:10.179842 19872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:10.180007 19872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:10.181159 19872 hybrid_clock.cc:648] HybridClock initialized: now 1790291470181145 us; error 33 us; skew 500 ppm
Sep 24 23:11:10 dist-test-slave-m0z6 krb5kdc[19174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291470, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:10.182889 19872 init.cc:377] Logged in from keytab as oryx/127.12.212.60@KRBTEST.COM (short username oryx)
I20260924 23:11:10.183171 19872 webserver.cc:460] Webserver started at http://127.12.212.60:38219/ using document root <none> and password file <none>
I20260924 23:11:10.183318 19872 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:10.183351 19872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:10.184120 19872 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:10.184739 19986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:10.184960 19872 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:10.185012 19872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/wal
uuid: "624c4e6f108b4c828263befbc5ae6c5d"
format_stamp: "Formatted at 2026-09-24 23:11:10 on dist-test-slave-m0z6"
server_key: "2388f5238bec870f5eda7658789049ed"
server_key_iv: "8c7f01f382cdd818b9af9cb73f90b273"
server_key_version: "encryptionkey@0"
I20260924 23:11:10.185076 19872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:10.193449 19872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:10.195544 19872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:10.198554 19872 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.60:45965
I20260924 23:11:10.198992 19872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/master-2/data/info.pb
I20260924 23:11:10.199327 20038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.60:45965 every 8 connection(s)
I20260924 23:11:10.200124 20039 sys_catalog.cc:263] Verifying existing consensus state
I20260924 23:11:10.200618 20039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: Bootstrap starting.
I20260924 23:11:10.203871 20039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 23:11:10.204098 20039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: Bootstrap complete.
I20260924 23:11:10.204334 20039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: NON_VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: true } }
I20260924 23:11:10.204439 20039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 624c4e6f108b4c828263befbc5ae6c5d, State: Initialized, Role: LEARNER
I20260924 23:11:10.204479 20039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: NON_VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: true } }
I20260924 23:11:10.204689 20039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:11:10.204761 20042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: NON_VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: true } } }
I20260924 23:11:10.204962 20042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [sys.catalog]: This master's current role is: LEARNER
I20260924 23:11:10.206162 20053 catalog_manager.cc:1259] Loaded cluster ID: 95821273abe2409e9ab3091d80765996
I20260924 23:11:10.206277 20053 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: loading cluster ID for follower catalog manager: success
I20260924 23:11:10.208144 20053 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: acquiring CA information for follower catalog manager: success
I20260924 23:11:10.208544 20053 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260924 23:11:10.227367 19965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:10.227610 19965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:10.227648 19965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:10.228960 19965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:10.229028 19965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:10.229070 19965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:11:10.230868 19965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.1.keytab
--principal=oryx/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:33293
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=36737
--tserver_master_addrs=127.12.212.62:37579,127.12.212.61:41905,127.12.212.60:45965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:10.231151 19965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:10.231400 19965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:10.237782 20059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:10.237785 20058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:10.238075 19965 server_base.cc:1034] running on GCE node
W20260924 23:11:10.238157 20061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:10.238397 19965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:10.238665 19965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:10.239878 19965 hybrid_clock.cc:648] HybridClock initialized: now 1790291470239857 us; error 39 us; skew 500 ppm
Sep 24 23:11:10 dist-test-slave-m0z6 krb5kdc[19174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291470, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:10.242652 19965 init.cc:377] Logged in from keytab as oryx/127.12.212.1@KRBTEST.COM (short username oryx)
I20260924 23:11:10.243148 19965 webserver.cc:460] Webserver started at http://127.12.212.1:36737/ using document root <none> and password file <none>
I20260924 23:11:10.243405 19965 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:10.243475 19965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:10.244926 19965 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:11:10.245501 20068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:10.245653 19965 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:10.245790 19965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "5816efe62c724db78e74db9e416e81ea"
format_stamp: "Formatted at 2026-09-24 23:11:09 on dist-test-slave-m0z6"
server_key: "8502ef28c88ae01c6a22a909e374d362"
server_key_iv: "17b7a2c65d93700210a670c01460c194"
server_key_version: "encryptionkey@0"
I20260924 23:11:10.246138 19965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:10.253926 19965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:10.256363 19965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:10.256513 19965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:10.256758 19965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:10.257058 19965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:10.257090 19965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:10.257120 19965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:10.257138 19965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:10.263355 19965 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:33293
I20260924 23:11:10.263437 20181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:33293 every 8 connection(s)
I20260924 23:11:10.263782 19965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:11:10.265173 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 19965
I20260924 23:11:10.265329 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 19415
I20260924 23:11:10.272130 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:38875
--local_ip_for_outbound_sockets=127.12.212.2
--tserver_master_addrs=127.12.212.62:37579
--webserver_port=42699
--webserver_interface=127.12.212.2
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.2.keytab
--principal=oryx/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.212.62:37579,127.12.212.61:41905
--tserver_master_addrs=127.12.212.62:37579,127.12.212.61:41905,127.12.212.60:45965 with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 23:11:10 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291470, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM
Sep 24 23:11:10 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291470, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.61@KRBTEST.COM
Sep 24 23:11:10 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291470, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.60@KRBTEST.COM
I20260924 23:11:10.276952 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.265182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:45939 (local address 127.12.212.62:37579)
0924 23:11:10.265223 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.265225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.265243 (+ 18us) server_negotiation.cc:407] Connection header received
0924 23:11:10.266395 (+ 1152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.266397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.266412 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.266465 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.267444 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.268243 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.269901 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.270114 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.270209 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.273120 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.273122 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.273122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.273131 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.275185 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.275768 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.275769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.275770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.275834 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.276131 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.276132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.276132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.276277 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.276333 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.276756 (+ 423us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.276847 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 23:11:10.277594 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.264948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:48687 (local address 127.12.212.61:41905)
0924 23:11:10.264997 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.264999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.265043 (+ 44us) server_negotiation.cc:407] Connection header received
0924 23:11:10.266434 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.266435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.266447 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.266494 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.267427 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.268223 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.271064 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.271378 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.271440 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.274844 (+ 3404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.274846 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.274846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.274855 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.276336 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.276712 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.276713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.276713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.276773 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.276958 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.276959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.276959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.277105 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.277152 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.277425 (+ 273us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.277505 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 23:11:10.277616 20189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.265019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.1:45939)
0924 23:11:10.265133 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.265143 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.266228 (+ 1085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.266570 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.266578 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.267061 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:10.267308 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.267314 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.268390 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.268391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.269726 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.269731 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.270260 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.270262 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.270479 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.271112 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:10.271124 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:10.272993 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:10.275274 (+ 2281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.275289 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.275292 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.275679 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.275901 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.275901 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.275901 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.276055 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.276382 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:10.276391 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:10.276591 (+ 200us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.277069 (+ 478us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.277357 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":18,"thread_start_us":61,"threads_started":1}
I20260924 23:11:10.278118 20188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.264846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:41905 (local address 127.12.212.1:48687)
0924 23:11:10.264985 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.264994 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.266198 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.266570 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.266574 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.267045 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:10.267308 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.267315 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.269858 (+ 2543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.269859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.270647 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.270650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.271505 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.271506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.271581 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.272086 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:10.272091 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:10.274730 (+ 2639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:10.276430 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.276431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.276432 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.276628 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.276838 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.276839 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.276839 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.276894 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.277234 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:10.277235 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:10.277279 (+ 44us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.277892 (+ 613us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.278017 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":22,"thread_start_us":74,"threads_started":1}
I20260924 23:11:10.278522 20184 heartbeater.cc:347] Connected to a master server at 127.12.212.61:41905
I20260924 23:11:10.278543 20182 heartbeater.cc:347] Connected to a master server at 127.12.212.62:37579
I20260924 23:11:10.278637 20184 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:10.278636 20182 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:10.278878 20182 heartbeater.cc:511] Master 127.12.212.62:37579 requested a full tablet report, sending...
I20260924 23:11:10.278878 20184 heartbeater.cc:511] Master 127.12.212.61:41905 requested a full tablet report, sending...
I20260924 23:11:10.279430 19219 ts_manager.cc:194] Re-registered known tserver with Master: 5816efe62c724db78e74db9e416e81ea (127.12.212.1:33293)
I20260924 23:11:10.279547 19818 ts_manager.cc:194] Registered new tserver with Master: 5816efe62c724db78e74db9e416e81ea (127.12.212.1:33293)
I20260924 23:11:10.279917 19219 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:45939
I20260924 23:11:10.281299 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.264657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:43993 (local address 127.12.212.60:45965)
0924 23:11:10.264784 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.264786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.264933 (+ 147us) server_negotiation.cc:407] Connection header received
0924 23:11:10.271431 (+ 6498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.271433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.271452 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.271501 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.271963 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.272966 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.274013 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.274346 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.274421 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.277461 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.277463 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.277463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.277472 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.279887 (+ 2415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.280341 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.280343 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.280343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.280417 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.280644 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.280645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.280646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.280788 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.280862 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.281099 (+ 237us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.281196 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":53,"threads_started":1}
I20260924 23:11:10.281456 20186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.264547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:45965 (local address 127.12.212.1:43993)
0924 23:11:10.264748 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.264800 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.271316 (+ 6516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.271574 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.271575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.271789 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:10.271853 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.271855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.273070 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.273071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.273881 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.273886 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.274518 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.274519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.274592 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.275260 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:10.275265 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:10.277354 (+ 2089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:10.279964 (+ 2610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.279965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.279965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.280241 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.280499 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.280499 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.280500 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.280568 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.280939 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:10.280940 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:10.280982 (+ 42us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.281048 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.281147 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":84,"threads_started":1}
I20260924 23:11:10.281941 20183 heartbeater.cc:347] Connected to a master server at 127.12.212.60:45965
I20260924 23:11:10.281991 20183 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:10.282075 20183 heartbeater.cc:511] Master 127.12.212.60:45965 requested a full tablet report, sending...
I20260924 23:11:10.282526 19994 ts_manager.cc:194] Registered new tserver with Master: 5816efe62c724db78e74db9e416e81ea (127.12.212.1:33293)
W20260924 23:11:10.358448 20190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:10.358678 20190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:10.358738 20190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:10.360024 20190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:10.360080 20190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:10.360108 20190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:11:10.361685 20190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.2.keytab
--principal=oryx/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:38875
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=42699
--tserver_master_addrs=127.12.212.62:37579,127.12.212.61:41905,127.12.212.60:45965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:10.361985 20190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:10.362221 20190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:11:10.367729 20190 server_base.cc:1034] running on GCE node
W20260924 23:11:10.367708 20195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:10.367708 20196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:10.367861 20198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:10.368187 20190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:10.368438 20190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:10.369602 20190 hybrid_clock.cc:648] HybridClock initialized: now 1790291470369585 us; error 30 us; skew 500 ppm
Sep 24 23:11:10 dist-test-slave-m0z6 krb5kdc[19174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291470, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:10.372304 20190 init.cc:377] Logged in from keytab as oryx/127.12.212.2@KRBTEST.COM (short username oryx)
I20260924 23:11:10.372749 20190 webserver.cc:460] Webserver started at http://127.12.212.2:42699/ using document root <none> and password file <none>
I20260924 23:11:10.372941 20190 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:10.372994 20190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:10.374507 20190 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:10.375443 20205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:10.375713 20190 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 23:11:10.375823 20190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "247c95b3bbae46d395cb9822ac4f9225"
format_stamp: "Formatted at 2026-09-24 23:11:09 on dist-test-slave-m0z6"
server_key: "063ebb8fe7be4db56c8b1f1b4e801bd2"
server_key_iv: "20b02f5f4a88f175c8ebdec7a5aa2ff4"
server_key_version: "encryptionkey@0"
I20260924 23:11:10.376170 20190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:10.388258 20190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:10.390432 20190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:10.390552 20190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:10.390779 20190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:10.391059 20190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:10.391088 20190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:10.391110 20190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:10.391127 20190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:10.396667 20190 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:38875
I20260924 23:11:10.396741 20318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:38875 every 8 connection(s)
I20260924 23:11:10.397054 20190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:11:10.398247 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 20190
I20260924 23:11:10.398411 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 19549
I20260924 23:11:10.404162 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:36715
--local_ip_for_outbound_sockets=127.12.212.3
--tserver_master_addrs=127.12.212.62:37579
--webserver_port=38549
--webserver_interface=127.12.212.3
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.3.keytab
--principal=oryx/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.212.62:37579,127.12.212.61:41905
--tserver_master_addrs=127.12.212.62:37579,127.12.212.61:41905,127.12.212.60:45965 with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 23:11:10 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291470, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM
Sep 24 23:11:10 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291470, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.60@KRBTEST.COM
Sep 24 23:11:10 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291470, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.61@KRBTEST.COM
I20260924 23:11:10.411291 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.397897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:39257 (local address 127.12.212.62:37579)
0924 23:11:10.397947 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.397949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.398159 (+ 210us) server_negotiation.cc:407] Connection header received
0924 23:11:10.399340 (+ 1181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.399342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.399355 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.399403 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.400141 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.400928 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.403259 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.403553 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.403593 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.407827 (+ 4234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.407829 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.407829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.407839 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.409536 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.410070 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.410071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.410072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.410134 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.410456 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.410457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.410457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.410623 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.410673 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.411090 (+ 417us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.411184 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 23:11:10.411288 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.397789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:48653 (local address 127.12.212.60:45965)
0924 23:11:10.397845 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.397848 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.398478 (+ 630us) server_negotiation.cc:407] Connection header received
0924 23:11:10.399459 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.399461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.399476 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.399524 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.400162 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.400922 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.402359 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.402563 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.402629 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.407827 (+ 5198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.407829 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.407829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.407838 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.410089 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.410487 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.410488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.410489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.410542 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.410768 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.410769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.410769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.410893 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.410933 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.411109 (+ 176us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.411189 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 23:11:10.411532 20324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.398187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:41905 (local address 127.12.212.2:50485)
0924 23:11:10.403609 (+ 5422us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.403621 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.403708 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.403887 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.403888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.404030 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:10.404091 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.404093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.405116 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.405117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.405865 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.405868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.406375 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.406376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.406440 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.406961 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:10.406966 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:10.408304 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:10.410057 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.410058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.410059 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.410318 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.410533 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.410534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.410534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.410592 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.410924 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:10.410924 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:10.410981 (+ 57us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.411062 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.411274 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5402,"thread_start_us":54,"threads_started":1}
I20260924 23:11:10.411635 20323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.397805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.2:39257)
0924 23:11:10.398038 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.398080 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.399235 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.399472 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.399476 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.399843 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:10.400047 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.400052 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.402364 (+ 2312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.402364 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.403143 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.403145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.403669 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.403670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.403731 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.405326 (+ 1595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:10.405331 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:10.407699 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:10.409630 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.409640 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.409643 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.409974 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.410195 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.410195 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.410196 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.410371 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.410736 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:10.410744 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:10.410928 (+ 184us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.411373 (+ 445us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.411493 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":155,"threads_started":1}
I20260924 23:11:10.412127 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.397805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:45965 (local address 127.12.212.2:48653)
0924 23:11:10.398409 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.398419 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.399247 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.399551 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.399552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.399853 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:10.400049 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.400055 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.401047 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.401048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.402256 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.402259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.402660 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.402661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.402875 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.403491 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:10.403501 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:10.407700 (+ 4199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:10.410178 (+ 2478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.410179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.410179 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.410403 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.410609 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.410609 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.410609 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.410663 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.410975 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:10.410976 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:10.411009 (+ 33us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.411937 (+ 928us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.412045 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":2,"thread_start_us":186,"threads_started":1}
I20260924 23:11:10.412171 20319 heartbeater.cc:347] Connected to a master server at 127.12.212.62:37579
I20260924 23:11:10.412343 20319 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:10.412536 20319 heartbeater.cc:511] Master 127.12.212.62:37579 requested a full tablet report, sending...
I20260924 23:11:10.412563 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.398278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:50485 (local address 127.12.212.61:41905)
0924 23:11:10.398353 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.398355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.403679 (+ 5324us) server_negotiation.cc:407] Connection header received
0924 23:11:10.403779 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.403780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.403794 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.403824 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.404195 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.404966 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.405958 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.406239 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.406301 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.408400 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.408401 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.408402 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.408409 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.409960 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.410401 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.410402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.410402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.410465 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.410656 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.410657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.410657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.410820 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.410896 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.412381 (+ 1485us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.412473 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 23:11:10.412885 20321 heartbeater.cc:347] Connected to a master server at 127.12.212.61:41905
I20260924 23:11:10.412937 20321 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:10.412941 19219 ts_manager.cc:194] Re-registered known tserver with Master: 247c95b3bbae46d395cb9822ac4f9225 (127.12.212.2:38875)
I20260924 23:11:10.413004 20321 heartbeater.cc:511] Master 127.12.212.61:41905 requested a full tablet report, sending...
I20260924 23:11:10.413240 20320 heartbeater.cc:347] Connected to a master server at 127.12.212.60:45965
I20260924 23:11:10.413321 20320 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:10.413322 19818 ts_manager.cc:194] Registered new tserver with Master: 247c95b3bbae46d395cb9822ac4f9225 (127.12.212.2:38875)
I20260924 23:11:10.413414 19219 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:39257
I20260924 23:11:10.413435 20320 heartbeater.cc:511] Master 127.12.212.60:45965 requested a full tablet report, sending...
I20260924 23:11:10.413738 19994 ts_manager.cc:194] Registered new tserver with Master: 247c95b3bbae46d395cb9822ac4f9225 (127.12.212.2:38875)
W20260924 23:11:10.490923 20326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:10.491146 20326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:10.491171 20326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:10.492424 20326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:10.492476 20326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:10.492504 20326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:11:10.494078 20326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.3.keytab
--principal=oryx/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:36715
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=38549
--tserver_master_addrs=127.12.212.62:37579,127.12.212.61:41905,127.12.212.60:45965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:10.494342 20326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:10.494611 20326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:10.499903 20334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:10.499954 20332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:10.499922 20331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:10.500317 20326 server_base.cc:1034] running on GCE node
I20260924 23:11:10.500523 20326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:10.500818 20326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:10.502005 20326 hybrid_clock.cc:648] HybridClock initialized: now 1790291470501988 us; error 31 us; skew 500 ppm
Sep 24 23:11:10 dist-test-slave-m0z6 krb5kdc[19174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291470, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:10.504815 20326 init.cc:377] Logged in from keytab as oryx/127.12.212.3@KRBTEST.COM (short username oryx)
I20260924 23:11:10.505328 20326 webserver.cc:460] Webserver started at http://127.12.212.3:38549/ using document root <none> and password file <none>
I20260924 23:11:10.505585 20326 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:10.505657 20326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:10.507272 20326 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:10.507965 20341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:10.508165 20326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:10.508315 20326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "0beb2bfd73a54e99a82ae77cfb709722"
format_stamp: "Formatted at 2026-09-24 23:11:09 on dist-test-slave-m0z6"
server_key: "6da8affe28750afd9b6daf990d65694c"
server_key_iv: "75bf8b41eeed64d0df1ee3bfd118dfe0"
server_key_version: "encryptionkey@0"
I20260924 23:11:10.508683 20326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:10.521354 20326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:10.523825 20326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:10.523954 20326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:10.524196 20326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:10.524533 20326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:10.524571 20326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:10.524602 20326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:10.524618 20326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:10.530897 20326 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:36715
I20260924 23:11:10.530967 20454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:36715 every 8 connection(s)
I20260924 23:11:10.531345 20326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
Sep 24 23:11:10 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291470, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.60@KRBTEST.COM
Sep 24 23:11:10 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291470, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM
Sep 24 23:11:10 dist-test-slave-m0z6 krb5kdc[19174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291470, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.61@KRBTEST.COM
I20260924 23:11:10.541152 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 20326
I20260924 23:11:10.543412 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.532511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:48077 (local address 127.12.212.62:37579)
0924 23:11:10.532561 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.532564 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.532616 (+ 52us) server_negotiation.cc:407] Connection header received
0924 23:11:10.533678 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.533680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.533695 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.533744 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.534715 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.535235 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.536469 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.536668 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.536728 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.540046 (+ 3318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.540049 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.540049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.540058 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.541607 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.542191 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.542192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.542193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.542266 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.542522 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.542523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.542524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.542725 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.542785 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.543217 (+ 432us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.543311 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 23:11:10.543918 20459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.532447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.3:48077)
0924 23:11:10.532570 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.532579 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.533535 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.533802 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.533807 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.534188 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:10.534490 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.534498 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.535327 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.535328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.536371 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.536374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.536756 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.536756 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.536915 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.537466 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:10.537477 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:10.539924 (+ 2447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:10.541713 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.541714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.541715 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.542080 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.542344 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.542345 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.542345 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.542446 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.542814 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:10.542823 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:10.543026 (+ 203us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.543559 (+ 533us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.543670 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":118,"thread_start_us":54,"threads_started":1}
I20260924 23:11:10.543932 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.532155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:46975 (local address 127.12.212.60:45965)
0924 23:11:10.532205 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.532207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.532600 (+ 393us) server_negotiation.cc:407] Connection header received
0924 23:11:10.533631 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.533633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.533647 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.533704 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.534621 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.535194 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.536385 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.536698 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.536741 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.539805 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.539808 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.539808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.539818 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.541495 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.542106 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.542108 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.542109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.542174 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.542525 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.542526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.542527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.542742 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.542803 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.543769 (+ 966us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.543855 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 23:11:10.544377 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.532166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:40175 (local address 127.12.212.61:41905)
0924 23:11:10.532279 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.532281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.532498 (+ 217us) server_negotiation.cc:407] Connection header received
0924 23:11:10.533711 (+ 1213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.533712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.533724 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.533775 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.535340 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.536144 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.537164 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.537364 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.537431 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.541433 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.541436 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.541436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.541446 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.543061 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.543545 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.543547 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.543547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.543611 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.543810 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.543812 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.543812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.543949 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.543993 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.544195 (+ 202us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.544273 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260924 23:11:10.544441 20455 heartbeater.cc:347] Connected to a master server at 127.12.212.62:37579
I20260924 23:11:10.544544 20455 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:10.544675 20461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.532160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:45965 (local address 127.12.212.3:46975)
0924 23:11:10.532540 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.532549 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.533497 (+ 948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.533802 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.533807 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.534188 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:10.534490 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.534498 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.535319 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.535320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.536275 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.536277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.536839 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.536840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.536919 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.537552 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:10.537558 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:10.539649 (+ 2091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:10.541580 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.541594 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.541598 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.542005 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.542252 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.542252 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.542253 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.542439 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.542827 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:10.542828 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:10.543026 (+ 198us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.543141 (+ 115us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.543393 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":52,"thread_start_us":173,"threads_started":1}
I20260924 23:11:10.544746 20460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.532162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:41905 (local address 127.12.212.3:40175)
0924 23:11:10.532339 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.532390 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.533477 (+ 1087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.533805 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.533809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.534182 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:10.534503 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.534506 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.536271 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.536273 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.537051 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.537053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.537459 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.537460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.537571 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.538096 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:10.538101 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:10.541296 (+ 3195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:10.543161 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.543162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.543163 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.543459 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.543683 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:10.543684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:10.543684 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:10.543741 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:10.544041 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:10.544042 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:10.544086 (+ 44us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.544507 (+ 421us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.544612 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":26,"thread_start_us":96,"threads_started":1}
I20260924 23:11:10.544754 20455 heartbeater.cc:511] Master 127.12.212.62:37579 requested a full tablet report, sending...
I20260924 23:11:10.545282 20457 heartbeater.cc:347] Connected to a master server at 127.12.212.61:41905
I20260924 23:11:10.545339 20457 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:10.545398 20457 heartbeater.cc:511] Master 127.12.212.61:41905 requested a full tablet report, sending...
I20260924 23:11:10.545392 19219 ts_manager.cc:194] Re-registered known tserver with Master: 0beb2bfd73a54e99a82ae77cfb709722 (127.12.212.3:36715)
I20260924 23:11:10.545704 19818 ts_manager.cc:194] Registered new tserver with Master: 0beb2bfd73a54e99a82ae77cfb709722 (127.12.212.3:36715)
I20260924 23:11:10.545902 19219 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:48077
I20260924 23:11:10.546034 20456 heartbeater.cc:347] Connected to a master server at 127.12.212.60:45965
I20260924 23:11:10.546082 20456 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:10.546144 20456 heartbeater.cc:511] Master 127.12.212.60:45965 requested a full tablet report, sending...
I20260924 23:11:10.546466 19994 ts_manager.cc:194] Registered new tserver with Master: 0beb2bfd73a54e99a82ae77cfb709722 (127.12.212.3:36715)
I20260924 23:11:10.553205 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.542763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52330 (local address 127.12.212.60:45965)
0924 23:11:10.543387 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.543388 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.543396 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:11:10.543418 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.543418 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.543430 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.543491 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.544305 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.546135 (+ 1830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.547698 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.548015 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.548083 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.549557 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.549559 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.549559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.549568 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.551601 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.552270 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.552272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.552272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.552398 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.552593 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.552593 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.552594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.552719 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.552815 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.552967 (+ 152us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.553056 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":69,"threads_started":1}
I20260924 23:11:10.554461 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.545040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46270 (local address 127.12.212.61:41905)
0924 23:11:10.545294 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.545297 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.546897 (+ 1600us) server_negotiation.cc:407] Connection header received
0924 23:11:10.546945 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.546946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.546956 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.547002 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.547675 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.548296 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.549453 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.549663 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.549705 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.551238 (+ 1533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.551240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.551241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.551255 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.553026 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.553421 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.553423 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.553423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.553486 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.553792 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.553793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.553794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.553952 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.554030 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.554271 (+ 241us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.554356 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231}
I20260924 23:11:10.555938 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.544798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35108 (local address 127.12.212.62:37579)
0924 23:11:10.544843 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.544845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.546415 (+ 1570us) server_negotiation.cc:407] Connection header received
0924 23:11:10.546447 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.546448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.546459 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.546520 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.547347 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.547951 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.549001 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.549311 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.549394 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.553049 (+ 3655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.553051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.553052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.553060 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.554850 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.555133 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.555134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.555134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.555183 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.555338 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.555338 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.555339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.555443 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.555523 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.555783 (+ 260us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.555865 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 23:11:10.557983 19219 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:11:10.559834 19219 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:11:10.568352 20474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.564358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:38875 (local address 127.0.0.1:56044)
0924 23:11:10.564503 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.564512 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.564552 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.565011 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.565012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.565015 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:10.565081 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.565083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.566066 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.566067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.567107 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.567110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.568076 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.568077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.568140 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.568145 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.568196 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.568256 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":104,"threads_started":1}
I20260924 23:11:10.568475 19777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.564268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:36715 (local address 127.0.0.1:53956)
0924 23:11:10.564304 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.564316 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.564355 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.565059 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.565060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.565062 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:10.565106 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.565108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.566366 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.566367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.567362 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.567365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.568238 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.568239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.568297 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.568300 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.568328 (+ 28us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.568370 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260924 23:11:10.568537 20477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.564594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56044 (local address 127.12.212.2:38875)
0924 23:11:10.564744 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.564750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.564761 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:11:10.564886 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.564889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.564913 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.565147 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:10.565207 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.565970 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.567210 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.567979 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.568054 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.568140 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.568213 (+ 73us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.568253 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
I20260924 23:11:10.568641 20478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.564610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41006 (local address 127.12.212.1:33293)
0924 23:11:10.564780 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.564785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.565005 (+ 220us) server_negotiation.cc:407] Connection header received
0924 23:11:10.565043 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.565045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.565060 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.565127 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:10.565331 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.566399 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.567537 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.568081 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.568147 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.568229 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.568528 (+ 299us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.568566 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260924 23:11:10.569401 20476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.564437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53956 (local address 127.12.212.3:36715)
0924 23:11:10.564886 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.564890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.564901 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:11:10.564938 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.564940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.564957 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.565061 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:10.565158 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.566267 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.567471 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.568154 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.569021 (+ 867us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.569111 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.569136 (+ 25us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.569178 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":92,"threads_started":1}
I20260924 23:11:10.568487 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.564386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:33293 (local address 127.0.0.1:41006)
0924 23:11:10.564889 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.564897 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.564943 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.565190 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.565191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.565192 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:10.565232 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.565234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.566487 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.566488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.567454 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.567456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.568261 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.568261 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.568330 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.568334 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.568376 (+ 42us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.568414 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":92,"threads_started":1}
I20260924 23:11:10.569535 20253 tablet_service.cc:1467] Processing CreateTablet for tablet 961ca6ca387240589e0782e81bbe4a59 (DEFAULT_TABLE table=test-table [id=7fd5d823bec841d781db969dcf26e2c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:10.569940 20253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 961ca6ca387240589e0782e81bbe4a59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:10.570114 20116 tablet_service.cc:1467] Processing CreateTablet for tablet 961ca6ca387240589e0782e81bbe4a59 (DEFAULT_TABLE table=test-table [id=7fd5d823bec841d781db969dcf26e2c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:10.570457 20116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 961ca6ca387240589e0782e81bbe4a59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:10.571156 20389 tablet_service.cc:1467] Processing CreateTablet for tablet 961ca6ca387240589e0782e81bbe4a59 (DEFAULT_TABLE table=test-table [id=7fd5d823bec841d781db969dcf26e2c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:10.571481 20389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 961ca6ca387240589e0782e81bbe4a59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:10.572860 20479 tablet_bootstrap.cc:492] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225: Bootstrap starting.
I20260924 23:11:10.573413 20479 tablet_bootstrap.cc:654] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:10.573694 20479 log.cc:824] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:10.574240 20481 tablet_bootstrap.cc:492] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea: Bootstrap starting.
I20260924 23:11:10.574633 20479 tablet_bootstrap.cc:492] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225: No bootstrap required, opened a new log
I20260924 23:11:10.574744 20479 ts_tablet_manager.cc:1397] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:11:10.574963 20481 tablet_bootstrap.cc:654] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:10.575270 20481 log.cc:824] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:10.576057 20481 tablet_bootstrap.cc:492] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea: No bootstrap required, opened a new log
I20260924 23:11:10.576136 20481 ts_tablet_manager.cc:1397] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:11:10.576784 20479 raft_consensus.cc:348] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } } peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } }
I20260924 23:11:10.577014 20479 raft_consensus.cc:374] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:10.577055 20479 raft_consensus.cc:729] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 247c95b3bbae46d395cb9822ac4f9225, State: Initialized, Role: FOLLOWER
I20260924 23:11:10.577211 20479 consensus_queue.cc:260] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } } peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } }
I20260924 23:11:10.577526 20479 ts_tablet_manager.cc:1428] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:11:10.577590 20319 heartbeater.cc:503] Master 127.12.212.62:37579 was elected leader, sending a full tablet report...
I20260924 23:11:10.577769 20482 tablet_bootstrap.cc:492] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722: Bootstrap starting.
I20260924 23:11:10.578562 20482 tablet_bootstrap.cc:654] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:10.578958 20482 log.cc:824] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:10.579564 20481 raft_consensus.cc:348] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } } peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } }
I20260924 23:11:10.579763 20481 raft_consensus.cc:374] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:10.579815 20481 raft_consensus.cc:729] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5816efe62c724db78e74db9e416e81ea, State: Initialized, Role: FOLLOWER
I20260924 23:11:10.579982 20481 consensus_queue.cc:260] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } } peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } }
I20260924 23:11:10.580173 20482 tablet_bootstrap.cc:492] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722: No bootstrap required, opened a new log
I20260924 23:11:10.580255 20482 ts_tablet_manager.cc:1397] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 23:11:10.580271 20481 ts_tablet_manager.cc:1428] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260924 23:11:10.580291 20182 heartbeater.cc:503] Master 127.12.212.62:37579 was elected leader, sending a full tablet report...
I20260924 23:11:10.582351 20482 raft_consensus.cc:348] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } } peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } }
I20260924 23:11:10.582523 20482 raft_consensus.cc:374] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:10.582545 20482 raft_consensus.cc:729] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0beb2bfd73a54e99a82ae77cfb709722, State: Initialized, Role: FOLLOWER
I20260924 23:11:10.582652 20482 consensus_queue.cc:260] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } } peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } }
I20260924 23:11:10.582970 20482 ts_tablet_manager.cc:1428] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:11:10.583019 20455 heartbeater.cc:503] Master 127.12.212.62:37579 was elected leader, sending a full tablet report...
W20260924 23:11:10.648307 20322 tablet.cc:2369] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:11:10.659811 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.657128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52336 (local address 127.12.212.60:45965)
0924 23:11:10.657193 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.657195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.657205 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:11:10.657330 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.657331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.657355 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.657415 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:10.657571 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.658078 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.659048 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.659459 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.659527 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.659621 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.659681 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.659733 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 23:11:10.659811 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.657009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:45965 (local address 127.0.0.1:52336)
0924 23:11:10.657104 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.657116 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.657249 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.657442 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.657443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.657449 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:10.657505 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.657508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.658167 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.658168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.658937 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.658941 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.659547 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.659548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.659612 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.659615 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.659670 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.659729 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260924 23:11:10.660386 20010 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260924 23:11:10.661365 20042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 838e7664b2b344b980d6324a8a2e3ec4. Latest consensus state: current_term: 1 leader_uuid: "838e7664b2b344b980d6324a8a2e3ec4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: NON_VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: true } } }
I20260924 23:11:10.661489 20042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [sys.catalog]: This master's current role is: LEARNER
W20260924 23:11:10.765784 20185 tablet.cc:2369] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 23:11:10.782222 20458 tablet.cc:2369] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:11:10.863524 20484 raft_consensus.cc:482] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:11:10.863667 20484 raft_consensus.cc:504] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } } peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } }
I20260924 23:11:10.864085 20484 leader_election.cc:288] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0beb2bfd73a54e99a82ae77cfb709722 (127.12.212.3:36715), 5816efe62c724db78e74db9e416e81ea (127.12.212.1:33293)
I20260924 23:11:10.867941 20476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.864401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:58427 (local address 127.12.212.3:36715)
0924 23:11:10.864455 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.864457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.864469 (+ 12us) server_negotiation.cc:407] Connection header received
0924 23:11:10.864501 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.864501 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.864510 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.864570 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:10.864832 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.865545 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.867139 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.867571 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.867640 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.867651 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.867830 (+ 179us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.867875 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 23:11:10.867941 20323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.864319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:36715 (local address 127.12.212.2:58427)
0924 23:11:10.864411 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.864419 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.864447 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.864582 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.864583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.864584 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:10.864675 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.864677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.865676 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.865678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.866997 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.867000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.867680 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.867681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.867752 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.867756 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.867802 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.867858 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":90}
I20260924 23:11:10.868052 20478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.864402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:37811 (local address 127.12.212.1:33293)
0924 23:11:10.864455 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.864457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.864469 (+ 12us) server_negotiation.cc:407] Connection header received
0924 23:11:10.864500 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.864501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.864510 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.864570 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:10.864780 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.865578 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.867138 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.867638 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.867719 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.867728 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.867933 (+ 205us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.867982 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 23:11:10.868083 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.864315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:33293 (local address 127.12.212.2:37811)
0924 23:11:10.864380 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:10.864392 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:10.864444 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:10.864576 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:10.864577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:10.864579 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:10.864675 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.864677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.865679 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.865679 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.866997 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:10.867000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.867780 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.867781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:10.867850 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.867853 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:11:10.867901 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 23:11:10.867949 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":91}
I20260924 23:11:10.868389 20136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "961ca6ca387240589e0782e81bbe4a59" candidate_uuid: "247c95b3bbae46d395cb9822ac4f9225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5816efe62c724db78e74db9e416e81ea" is_pre_election: true
I20260924 23:11:10.868583 20136 raft_consensus.cc:2454] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 247c95b3bbae46d395cb9822ac4f9225 in term 0.
I20260924 23:11:10.868536 20409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "961ca6ca387240589e0782e81bbe4a59" candidate_uuid: "247c95b3bbae46d395cb9822ac4f9225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0beb2bfd73a54e99a82ae77cfb709722" is_pre_election: true
I20260924 23:11:10.868690 20409 raft_consensus.cc:2454] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 247c95b3bbae46d395cb9822ac4f9225 in term 0.
I20260924 23:11:10.868906 20208 leader_election.cc:302] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 247c95b3bbae46d395cb9822ac4f9225, 5816efe62c724db78e74db9e416e81ea; no voters:
I20260924 23:11:10.869065 20484 raft_consensus.cc:2781] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:11:10.869122 20484 raft_consensus.cc:482] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:11:10.869148 20484 raft_consensus.cc:3037] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:10.869887 20484 raft_consensus.cc:504] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } } peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } }
I20260924 23:11:10.870030 20484 leader_election.cc:288] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [CANDIDATE]: Term 1 election: Requested vote from peers 0beb2bfd73a54e99a82ae77cfb709722 (127.12.212.3:36715), 5816efe62c724db78e74db9e416e81ea (127.12.212.1:33293)
I20260924 23:11:10.870239 20136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "961ca6ca387240589e0782e81bbe4a59" candidate_uuid: "247c95b3bbae46d395cb9822ac4f9225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5816efe62c724db78e74db9e416e81ea"
I20260924 23:11:10.870396 20136 raft_consensus.cc:3037] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:10.871042 20409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "961ca6ca387240589e0782e81bbe4a59" candidate_uuid: "247c95b3bbae46d395cb9822ac4f9225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0beb2bfd73a54e99a82ae77cfb709722"
I20260924 23:11:10.871143 20409 raft_consensus.cc:3037] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:10.871371 20136 raft_consensus.cc:2454] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 247c95b3bbae46d395cb9822ac4f9225 in term 1.
I20260924 23:11:10.871585 20208 leader_election.cc:302] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 247c95b3bbae46d395cb9822ac4f9225, 5816efe62c724db78e74db9e416e81ea; no voters:
I20260924 23:11:10.871721 20484 raft_consensus.cc:2781] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:10.871897 20484 raft_consensus.cc:686] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [term 1 LEADER]: Becoming Leader. State: Replica: 247c95b3bbae46d395cb9822ac4f9225, State: Running, Role: LEADER
I20260924 23:11:10.872044 20409 raft_consensus.cc:2454] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 247c95b3bbae46d395cb9822ac4f9225 in term 1.
I20260924 23:11:10.872030 20484 consensus_queue.cc:237] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } } peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } }
I20260924 23:11:10.873010 19219 catalog_manager.cc:5495] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 reported cstate change: term changed from 0 to 1, leader changed from <none> to 247c95b3bbae46d395cb9822ac4f9225 (127.12.212.2). New cstate: current_term: 1 leader_uuid: "247c95b3bbae46d395cb9822ac4f9225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } health_report { overall_health: HEALTHY } } }
I20260924 23:11:10.873728 19779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "838e7664b2b344b980d6324a8a2e3ec4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: NON_VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: true } } }
I20260924 23:11:10.873831 19779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:10.874781 19779 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4: attempting to promote NON_VOTER 624c4e6f108b4c828263befbc5ae6c5d to VOTER
I20260924 23:11:10.874898 19779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } }
I20260924 23:11:10.875371 19828 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 1 FOLLOWER]: Refusing update from remote peer 838e7664b2b344b980d6324a8a2e3ec4: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260924 23:11:10.875389 20010 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 1 LEARNER]: Refusing update from remote peer 838e7664b2b344b980d6324a8a2e3ec4: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260924 23:11:10.875574 20494 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 23:11:10.875701 20495 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 23:11:10.876416 19780 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 624c4e6f108b4c828263befbc5ae6c5d (127.12.212.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } } }
I20260924 23:11:10.876556 20010 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 624c4e6f108b4c828263befbc5ae6c5d (127.12.212.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } } }
I20260924 23:11:10.876655 19828 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 624c4e6f108b4c828263befbc5ae6c5d (127.12.212.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } } }
I20260924 23:11:10.877254 19780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "838e7664b2b344b980d6324a8a2e3ec4" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } } }
I20260924 23:11:10.877334 19780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:10.877391 19780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "838e7664b2b344b980d6324a8a2e3ec4" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } } }
I20260924 23:11:10.877439 19780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:10.877528 20042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "838e7664b2b344b980d6324a8a2e3ec4" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } } }
I20260924 23:11:10.877596 20042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:11:10.877431 20494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "838e7664b2b344b980d6324a8a2e3ec4" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } } }
I20260924 23:11:10.877780 20494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:10.877753 20497 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "838e7664b2b344b980d6324a8a2e3ec4" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } } }
I20260924 23:11:10.877913 20497 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:11:10.930284 20477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.927114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56046 (local address 127.12.212.2:38875)
0924 23:11:10.927204 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.927206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.927215 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:11:10.927252 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.927253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.927273 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.927403 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:11:10.927651 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.928130 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.929163 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.929396 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.929470 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.929805 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:11:10.929934 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:11:10.930017 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.930166 (+ 149us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.930191 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260924 23:11:10.932359 20409 raft_consensus.cc:1261] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722 [term 1 FOLLOWER]: Refusing update from remote peer 247c95b3bbae46d395cb9822ac4f9225: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:10.932407 20136 raft_consensus.cc:1261] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea [term 1 FOLLOWER]: Refusing update from remote peer 247c95b3bbae46d395cb9822ac4f9225: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:10.932718 20484 consensus_queue.cc:1035] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:10.932876 20492 consensus_queue.cc:1035] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:10.935586 20499 mvcc.cc:205] Tried to move back new op lower bound from 7333033864935989248 to 7333033864692686848. Current Snapshot: MvccSnapshot[applied={T|T < 7333033864935989248}]
I20260924 23:11:10.936765 20501 mvcc.cc:205] Tried to move back new op lower bound from 7333033864935989248 to 7333033864692686848. Current Snapshot: MvccSnapshot[applied={T|T < 7333033864935989248}]
I20260924 23:11:10.937613 20500 mvcc.cc:205] Tried to move back new op lower bound from 7333033864935989248 to 7333033864692686848. Current Snapshot: MvccSnapshot[applied={T|T < 7333033864935989248}]
I20260924 23:11:10.939061 19218 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:11:10.939157 19218 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:11:10.940538 19218 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 838e7664b2b344b980d6324a8a2e3ec4: Sending DeleteTablet for 3 replicas of tablet 961ca6ca387240589e0782e81bbe4a59
I20260924 23:11:10.940971 20116 tablet_service.cc:1514] Processing DeleteTablet for tablet 961ca6ca387240589e0782e81bbe4a59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:10 UTC) from {username='oryx', principal='oryx/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:41006
I20260924 23:11:10.941043 20389 tablet_service.cc:1514] Processing DeleteTablet for tablet 961ca6ca387240589e0782e81bbe4a59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:10 UTC) from {username='oryx', principal='oryx/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:53956
I20260924 23:11:10.941038 20253 tablet_service.cc:1514] Processing DeleteTablet for tablet 961ca6ca387240589e0782e81bbe4a59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:10 UTC) from {username='oryx', principal='oryx/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:56044
I20260924 23:11:10.941269 20510 tablet_replica.cc:330] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722: stopping tablet replica
I20260924 23:11:10.941314 20511 tablet_replica.cc:330] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225: stopping tablet replica
I20260924 23:11:10.941390 20510 raft_consensus.cc:2229] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:11:10.941419 20511 raft_consensus.cc:2229] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:11:10.941546 20511 raft_consensus.cc:2258] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:10.941510 20510 raft_consensus.cc:2258] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:10.941962 20511 ts_tablet_manager.cc:1905] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:10.942485 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 19187
I20260924 23:11:10.943773 20509 tablet_replica.cc:330] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea: stopping tablet replica
I20260924 23:11:10.943924 20509 raft_consensus.cc:2229] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:11:10.944067 20509 raft_consensus.cc:2258] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:10.944245 20511 ts_tablet_manager.cc:1918] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:11:10.944331 20511 log.cc:1196] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/961ca6ca387240589e0782e81bbe4a59
I20260924 23:11:10.944618 20511 ts_tablet_manager.cc:1939] T 961ca6ca387240589e0782e81bbe4a59 P 247c95b3bbae46d395cb9822ac4f9225: Deleting consensus metadata
I20260924 23:11:10.945525 20509 ts_tablet_manager.cc:1905] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:10.947775 20509 ts_tablet_manager.cc:1918] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:11:10.947876 20509 log.cc:1196] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-0/wal/wals/961ca6ca387240589e0782e81bbe4a59
I20260924 23:11:10.948148 20509 ts_tablet_manager.cc:1939] T 961ca6ca387240589e0782e81bbe4a59 P 5816efe62c724db78e74db9e416e81ea: Deleting consensus metadata
W20260924 23:11:10.949224 20207 connection.cc:535] server connection from 127.0.0.1:56044 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 23:11:10.949453 20069 connection.cc:535] server connection from 127.0.0.1:41006 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 23:11:10.950665 20510 ts_tablet_manager.cc:1905] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:10.952742 20510 ts_tablet_manager.cc:1918] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:11:10.952891 20510 log.cc:1196] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/961ca6ca387240589e0782e81bbe4a59
I20260924 23:11:10.953171 20510 ts_tablet_manager.cc:1939] T 961ca6ca387240589e0782e81bbe4a59 P 0beb2bfd73a54e99a82ae77cfb709722: Deleting consensus metadata
W20260924 23:11:10.953570 20342 connection.cc:441] server connection from 127.0.0.1:53956 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53956 (request call id 1) could send its response
I20260924 23:11:10.953737 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.942139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52352 (local address 127.12.212.60:45965)
0924 23:11:10.942178 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.942180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.945178 (+ 2998us) server_negotiation.cc:407] Connection header received
0924 23:11:10.945240 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.945241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.945254 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.945304 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.946141 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.946954 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.948452 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.948730 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.948774 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.950493 (+ 1719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.950495 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.950495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.950503 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.952296 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.952799 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.952801 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.952801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.952875 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.953092 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.953094 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.953095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.953309 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.953420 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.953552 (+ 132us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.953645 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 23:11:10.962471 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.955411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46276 (local address 127.12.212.61:41905)
0924 23:11:10.955456 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.955458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.955662 (+ 204us) server_negotiation.cc:407] Connection header received
0924 23:11:10.955808 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.955809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.955821 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.955881 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.956437 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.957204 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.958177 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.958515 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.958582 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.959933 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.959934 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.959934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.959942 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.961284 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.961656 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.961657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.961658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.961711 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.961904 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.961905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.961906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.962025 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.962114 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.962267 (+ 153us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.962357 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 23:11:10.962767 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:10.954984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52360 (local address 127.12.212.60:45965)
0924 23:11:10.955024 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:10.955026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:10.955156 (+ 130us) server_negotiation.cc:407] Connection header received
0924 23:11:10.955266 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:10.955266 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:10.955277 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:10.955325 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:10.956185 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.956783 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.957584 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:10.957882 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:10.957920 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:10.959515 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:10.959516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:10.959517 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:10.959525 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:10.961165 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.961616 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.961617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.961617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.961682 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:10.961905 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:10.961905 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:10.961906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:10.962050 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:10.962146 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:10.962531 (+ 385us) server_negotiation.cc:299] Negotiation successful
0924 23:11:10.962639 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 23:11:11.937358 20520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:11.937036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.1:43333)
0924 23:11:11.937206 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:11.937300 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
W20260924 23:11:11.937569 20182 heartbeater.cc:644] Failed to heartbeat to 127.12.212.62:37579 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
I20260924 23:11:11.938871 20522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:11.938622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.2:55055)
0924 23:11:11.938749 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:11.938822 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
I20260924 23:11:11.938884 20521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:11.938626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.3:48857)
0924 23:11:11.938786 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:11.938848 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":90,"threads_started":1}
W20260924 23:11:11.939132 20319 heartbeater.cc:644] Failed to heartbeat to 127.12.212.62:37579 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
W20260924 23:11:11.939158 20455 heartbeater.cc:644] Failed to heartbeat to 127.12.212.62:37579 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
I20260924 23:11:12.441454 20524 raft_consensus.cc:482] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 838e7664b2b344b980d6324a8a2e3ec4)
I20260924 23:11:12.441497 20523 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 838e7664b2b344b980d6324a8a2e3ec4)
I20260924 23:11:12.441612 20524 raft_consensus.cc:504] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } }
I20260924 23:11:12.441634 20523 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } }
I20260924 23:11:12.441922 20523 leader_election.cc:288] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 838e7664b2b344b980d6324a8a2e3ec4 (127.12.212.62:37579), 624c4e6f108b4c828263befbc5ae6c5d (127.12.212.60:45965)
I20260924 23:11:12.441897 20524 leader_election.cc:288] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 838e7664b2b344b980d6324a8a2e3ec4 (127.12.212.62:37579), cfc0464cc67a40de8e97ffd2dfb0c48b (127.12.212.61:41905)
I20260924 23:11:12.442396 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:12.442042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35522)
0924 23:11:12.442303 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:12.442360 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":58,"threads_started":1}
W20260924 23:11:12.442514 19803 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
I20260924 23:11:12.442425 20526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:12.442043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35520)
0924 23:11:12.442177 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:12.442246 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
W20260924 23:11:12.442794 19988 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
I20260924 23:11:12.443148 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:12.443043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35540)
0924 23:11:12.443093 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:12.443121 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260924 23:11:12.443322 19803 leader_election.cc:334] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 838e7664b2b344b980d6324a8a2e3ec4 (127.12.212.62:37579): Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
I20260924 23:11:12.443343 20526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:12.443151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35546)
0924 23:11:12.443288 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:12.443316 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":122,"threads_started":1}
W20260924 23:11:12.443506 19988 leader_election.cc:334] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 838e7664b2b344b980d6324a8a2e3ec4 (127.12.212.62:37579): Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
I20260924 23:11:12.446038 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:12.442131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:45965 (local address 127.0.0.1:52366)
0924 23:11:12.442401 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:12.442411 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:12.442476 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:12.442673 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:12.442674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:12.442676 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:12.442757 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:12.442759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:12.443718 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:12.443718 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:12.444931 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:12.444935 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:12.445747 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:12.445747 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:12.445816 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:12.445821 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 23:11:12.445881 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 23:11:12.445926 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":44,"threads_started":1}
I20260924 23:11:12.446035 20529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:12.442208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:41905 (local address 127.0.0.1:46278)
0924 23:11:12.442411 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:12.442419 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:12.442537 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:12.442741 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:12.442742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:12.442744 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:12.442782 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:12.442784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:12.443704 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:12.443705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:12.444931 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:12.444934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:12.445746 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:12.445747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:12.445816 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:12.445821 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 23:11:12.445881 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 23:11:12.445926 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":65,"threads_started":1}
I20260924 23:11:12.446038 20528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:12.442205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52366 (local address 127.12.212.60:45965)
0924 23:11:12.442358 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:12.442360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:12.442526 (+ 166us) server_negotiation.cc:407] Connection header received
0924 23:11:12.442551 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:12.442552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:12.442561 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:12.442624 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:12.442829 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:12.443620 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:12.445046 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:12.445632 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:12.445716 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:12.445729 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:12.445897 (+ 168us) server_negotiation.cc:299] Negotiation successful
0924 23:11:12.445940 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":40,"threads_started":1}
I20260924 23:11:12.446038 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:12.442358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46278 (local address 127.12.212.61:41905)
0924 23:11:12.442498 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:12.442500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:12.442509 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:11:12.442630 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:12.442631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:12.442638 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:12.442687 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:12.442848 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:12.443626 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:12.445046 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:12.445662 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:12.445722 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:12.445730 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:12.445897 (+ 167us) server_negotiation.cc:299] Negotiation successful
0924 23:11:12.445940 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":36,"threads_started":1}
I20260924 23:11:12.446406 19828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "624c4e6f108b4c828263befbc5ae6c5d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" is_pre_election: true
I20260924 23:11:12.446434 20010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "624c4e6f108b4c828263befbc5ae6c5d" is_pre_election: true
I20260924 23:11:12.446523 20010 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfc0464cc67a40de8e97ffd2dfb0c48b in term 1.
I20260924 23:11:12.446523 19828 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 624c4e6f108b4c828263befbc5ae6c5d in term 1.
I20260924 23:11:12.446679 19988 leader_election.cc:302] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 624c4e6f108b4c828263befbc5ae6c5d, cfc0464cc67a40de8e97ffd2dfb0c48b; no voters: 838e7664b2b344b980d6324a8a2e3ec4
I20260924 23:11:12.446686 19804 leader_election.cc:302] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 624c4e6f108b4c828263befbc5ae6c5d, cfc0464cc67a40de8e97ffd2dfb0c48b; no voters: 838e7664b2b344b980d6324a8a2e3ec4
I20260924 23:11:12.446807 20524 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 23:11:12.446817 20523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 23:11:12.446848 20523 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 838e7664b2b344b980d6324a8a2e3ec4)
I20260924 23:11:12.446848 20524 raft_consensus.cc:482] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 838e7664b2b344b980d6324a8a2e3ec4)
I20260924 23:11:12.446874 20524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 1 FOLLOWER]: Advancing to term 2
I20260924 23:11:12.446874 20523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 1 FOLLOWER]: Advancing to term 2
I20260924 23:11:12.447722 20523 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } }
I20260924 23:11:12.447722 20524 raft_consensus.cc:504] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } }
I20260924 23:11:12.447862 20524 leader_election.cc:288] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [CANDIDATE]: Term 2 election: Requested vote from peers 838e7664b2b344b980d6324a8a2e3ec4 (127.12.212.62:37579), cfc0464cc67a40de8e97ffd2dfb0c48b (127.12.212.61:41905)
I20260924 23:11:12.447873 20523 leader_election.cc:288] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [CANDIDATE]: Term 2 election: Requested vote from peers 838e7664b2b344b980d6324a8a2e3ec4 (127.12.212.62:37579), 624c4e6f108b4c828263befbc5ae6c5d (127.12.212.60:45965)
I20260924 23:11:12.448149 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:12.448008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35552)
0924 23:11:12.448085 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:12.448119 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260924 23:11:12.448159 20010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "624c4e6f108b4c828263befbc5ae6c5d"
I20260924 23:11:12.448236 20010 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cfc0464cc67a40de8e97ffd2dfb0c48b in current term 2: Already voted for candidate 624c4e6f108b4c828263befbc5ae6c5d in this term.
I20260924 23:11:12.448248 19828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "624c4e6f108b4c828263befbc5ae6c5d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b"
I20260924 23:11:12.448316 19828 raft_consensus.cc:2379] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 624c4e6f108b4c828263befbc5ae6c5d in current term 2: Already voted for candidate cfc0464cc67a40de8e97ffd2dfb0c48b in this term.
I20260924 23:11:12.448320 20529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:12.448114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35568)
0924 23:11:12.448164 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:12.448238 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 23:11:12.448474 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:12.448398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35584)
0924 23:11:12.448429 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:12.448455 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260924 23:11:12.448606 19803 leader_election.cc:334] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 838e7664b2b344b980d6324a8a2e3ec4 (127.12.212.62:37579): Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
I20260924 23:11:12.448657 19803 leader_election.cc:302] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cfc0464cc67a40de8e97ffd2dfb0c48b; no voters: 624c4e6f108b4c828263befbc5ae6c5d, 838e7664b2b344b980d6324a8a2e3ec4
I20260924 23:11:12.448753 20523 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 23:11:12.448788 20529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:12.448667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35586)
0924 23:11:12.448739 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:12.448763 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260924 23:11:12.448854 19988 leader_election.cc:334] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 838e7664b2b344b980d6324a8a2e3ec4 (127.12.212.62:37579): Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
I20260924 23:11:12.448920 19988 leader_election.cc:302] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 624c4e6f108b4c828263befbc5ae6c5d; no voters: 838e7664b2b344b980d6324a8a2e3ec4, cfc0464cc67a40de8e97ffd2dfb0c48b
I20260924 23:11:12.448987 20524 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 23:11:12.938133 20534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:12.937900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.1:39665)
0924 23:11:12.938046 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:12.938089 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
I20260924 23:11:12.939634 20535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:12.939430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.3:42871)
0924 23:11:12.939552 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:12.939591 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260924 23:11:12.939656 20536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:12.939451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.2:50361)
0924 23:11:12.939593 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:12.939621 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":52,"threads_started":1}
I20260924 23:11:13.938890 20537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:13.938627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.1:39569)
0924 23:11:13.938793 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:13.938834 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":98,"threads_started":1}
I20260924 23:11:13.940296 20538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:13.940078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.2:44915)
0924 23:11:13.940202 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:13.940246 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":53,"threads_started":1}
I20260924 23:11:13.940459 20539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:13.940179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.3:56693)
0924 23:11:13.940309 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:13.940347 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
I20260924 23:11:14.512107 20540 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:11:14.512204 20540 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } }
I20260924 23:11:14.512388 20540 leader_election.cc:288] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 838e7664b2b344b980d6324a8a2e3ec4 (127.12.212.62:37579), 624c4e6f108b4c828263befbc5ae6c5d (127.12.212.60:45965)
I20260924 23:11:14.512639 20010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "624c4e6f108b4c828263befbc5ae6c5d" is_pre_election: true
I20260924 23:11:14.512702 20541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.512512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35884)
0924 23:11:14.512619 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.512655 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":48,"threads_started":1}
I20260924 23:11:14.512727 20010 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfc0464cc67a40de8e97ffd2dfb0c48b in term 2.
I20260924 23:11:14.512874 19804 leader_election.cc:302] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 624c4e6f108b4c828263befbc5ae6c5d, cfc0464cc67a40de8e97ffd2dfb0c48b; no voters:
I20260924 23:11:14.512959 20540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 23:11:14.513005 20540 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:11:14.513026 20540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 2 FOLLOWER]: Advancing to term 3
I20260924 23:11:14.513087 20541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.512992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35888)
0924 23:11:14.513033 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.513066 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260924 23:11:14.513152 19803 leader_election.cc:334] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 838e7664b2b344b980d6324a8a2e3ec4 (127.12.212.62:37579): Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
I20260924 23:11:14.513774 20540 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } }
I20260924 23:11:14.513898 20540 leader_election.cc:288] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [CANDIDATE]: Term 3 election: Requested vote from peers 838e7664b2b344b980d6324a8a2e3ec4 (127.12.212.62:37579), 624c4e6f108b4c828263befbc5ae6c5d (127.12.212.60:45965)
I20260924 23:11:14.514120 20541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.514000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35904)
0924 23:11:14.514059 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.514094 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 23:11:14.514200 20010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "624c4e6f108b4c828263befbc5ae6c5d"
I20260924 23:11:14.514292 20010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 2 FOLLOWER]: Advancing to term 3
I20260924 23:11:14.514413 20541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.514310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35912)
0924 23:11:14.514343 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.514371 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260924 23:11:14.514485 19803 leader_election.cc:334] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 838e7664b2b344b980d6324a8a2e3ec4 (127.12.212.62:37579): Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
I20260924 23:11:14.515018 20010 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfc0464cc67a40de8e97ffd2dfb0c48b in term 3.
I20260924 23:11:14.515201 19804 leader_election.cc:302] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 624c4e6f108b4c828263befbc5ae6c5d, cfc0464cc67a40de8e97ffd2dfb0c48b; no voters: 838e7664b2b344b980d6324a8a2e3ec4
I20260924 23:11:14.515319 20540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 3 FOLLOWER]: Leader election won for term 3
I20260924 23:11:14.515431 20540 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [term 3 LEADER]: Becoming Leader. State: Replica: cfc0464cc67a40de8e97ffd2dfb0c48b, State: Running, Role: LEADER
I20260924 23:11:14.515501 20540 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } }
I20260924 23:11:14.515868 20543 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfc0464cc67a40de8e97ffd2dfb0c48b. Latest consensus state: current_term: 3 leader_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } } }
I20260924 23:11:14.515971 20543 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:14.516160 20545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:11:14.516577 20545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:11:14.516637 20545 catalog_manager.cc:1259] Loaded cluster ID: 95821273abe2409e9ab3091d80765996
I20260924 23:11:14.516664 20545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:11:14.516848 20545 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:11:14.516909 20545 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: Loaded TSK: 0
I20260924 23:11:14.517035 20545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:11:14.546672 19817 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:11:14.548480 19817 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:11:14.549595 20541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.549471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35932)
0924 23:11:14.549525 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.549564 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 23:11:14.549655 20010 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [term 3 FOLLOWER]: Refusing update from remote peer cfc0464cc67a40de8e97ffd2dfb0c48b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260924 23:11:14.549988 20541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.549895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.0.0.1:35936)
0924 23:11:14.549931 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.549963 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260924 23:11:14.549927 20540 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [LEADER]: Connected to new peer: Peer: permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260924 23:11:14.550176 19803 consensus_peers.cc:487] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b -> Peer 838e7664b2b344b980d6324a8a2e3ec4 (127.12.212.62:37579): Couldn't send request to peer 838e7664b2b344b980d6324a8a2e3ec4. Status: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 23:11:14.550730 20547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfc0464cc67a40de8e97ffd2dfb0c48b. Latest consensus state: current_term: 3 leader_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } } }
I20260924 23:11:14.550819 20547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:11:14.551291 20546 mvcc.cc:205] Tried to move back new op lower bound from 7333033879753248768 to 7333033879616131072. Current Snapshot: MvccSnapshot[applied={T|T < 7333033879753248768}]
I20260924 23:11:14.551374 20543 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } } }
I20260924 23:11:14.551460 20543 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:14.551515 20548 mvcc.cc:205] Tried to move back new op lower bound from 7333033879753248768 to 7333033879616131072. Current Snapshot: MvccSnapshot[applied={T|T < 7333033879753248768}]
I20260924 23:11:14.551914 20547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "838e7664b2b344b980d6324a8a2e3ec4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37579 } } peers { permanent_uuid: "cfc0464cc67a40de8e97ffd2dfb0c48b" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 41905 } attrs { promote: false } } peers { permanent_uuid: "624c4e6f108b4c828263befbc5ae6c5d" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 45965 } attrs { promote: false } } }
I20260924 23:11:14.552009 20547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 624c4e6f108b4c828263befbc5ae6c5d [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:11:14.556785 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.553469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:36715 (local address 127.0.0.1:53968)
0924 23:11:14.553736 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.553746 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:14.553795 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:14.554055 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:14.554055 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:14.554057 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:14.554087 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:14.554088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:14.554751 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:14.554752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:14.555925 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:14.555928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:14.556536 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:14.556537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:14.556597 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:14.556600 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:11:14.556644 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 23:11:14.556691 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":93,"threads_started":1}
I20260924 23:11:14.556777 20555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.553617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53968 (local address 127.12.212.3:36715)
0924 23:11:14.553935 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:14.553937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:14.553944 (+ 7us) server_negotiation.cc:407] Connection header received
0924 23:11:14.553968 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:14.553969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:14.553977 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:14.554050 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:14.554160 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:14.554655 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:14.556001 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:14.556451 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:14.556511 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:14.556526 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:14.556665 (+ 139us) server_negotiation.cc:299] Negotiation successful
0924 23:11:14.556705 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":59,"threads_started":1}
I20260924 23:11:14.556938 20541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.553364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:38875 (local address 127.0.0.1:56062)
0924 23:11:14.553417 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.553428 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:14.553477 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:14.553905 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:14.553906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:14.553907 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:14.553959 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:14.553960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:14.554919 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:14.554920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:14.555973 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:14.555974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:14.556717 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:14.556718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:14.556780 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:14.556784 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:11:14.556812 (+ 28us) client_negotiation.cc:240] Negotiation successful
0924 23:11:14.556860 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 23:11:14.557049 20389 tablet_service.cc:1467] Processing CreateTablet for tablet 1e3c933b89c74fd68d67737c64aadd07 (DEFAULT_TABLE table=test-table [id=614db433606b4aa19feb3c0a3d747d2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:14.557188 20389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e3c933b89c74fd68d67737c64aadd07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:14.557255 20554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.553621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56062 (local address 127.12.212.2:38875)
0924 23:11:14.553732 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:14.553734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:14.553742 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:11:14.553769 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:14.553771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:14.553782 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:14.553866 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:14.554049 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:14.554841 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:14.556055 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:14.556610 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:14.557100 (+ 490us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:14.557117 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:14.557136 (+ 19us) server_negotiation.cc:299] Negotiation successful
0924 23:11:14.557179 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":51,"threads_started":1}
I20260924 23:11:14.557443 20253 tablet_service.cc:1467] Processing CreateTablet for tablet 1e3c933b89c74fd68d67737c64aadd07 (DEFAULT_TABLE table=test-table [id=614db433606b4aa19feb3c0a3d747d2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:14.557557 20253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e3c933b89c74fd68d67737c64aadd07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:14.557602 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.553409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:33293 (local address 127.0.0.1:41012)
0924 23:11:14.553727 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.553736 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:14.553781 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:14.554269 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:14.554270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:14.554272 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:14.554316 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:14.554317 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:14.555207 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:14.555208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:14.556068 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:14.556071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:14.556821 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:14.556822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:14.557433 (+ 611us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:14.557438 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 23:11:14.557496 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 23:11:14.557539 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":90,"threads_started":1}
I20260924 23:11:14.557648 20556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.553618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41012 (local address 127.12.212.1:33293)
0924 23:11:14.554125 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:14.554127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:14.554133 (+ 6us) server_negotiation.cc:407] Connection header received
0924 23:11:14.554157 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:14.554158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:14.554169 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:14.554223 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:14.554379 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:14.555127 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:14.556159 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:14.556723 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:14.556781 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:14.556794 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:14.557530 (+ 736us) server_negotiation.cc:299] Negotiation successful
0924 23:11:14.557582 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":60,"threads_started":1}
I20260924 23:11:14.558768 20557 tablet_bootstrap.cc:492] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722: Bootstrap starting.
I20260924 23:11:14.559007 20558 tablet_bootstrap.cc:492] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225: Bootstrap starting.
I20260924 23:11:14.559191 20116 tablet_service.cc:1467] Processing CreateTablet for tablet 1e3c933b89c74fd68d67737c64aadd07 (DEFAULT_TABLE table=test-table [id=614db433606b4aa19feb3c0a3d747d2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:14.559289 20116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e3c933b89c74fd68d67737c64aadd07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:14.559362 20558 tablet_bootstrap.cc:654] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:14.559996 20558 tablet_bootstrap.cc:492] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225: No bootstrap required, opened a new log
I20260924 23:11:14.560055 20558 ts_tablet_manager.cc:1397] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:14.560177 20558 raft_consensus.cc:348] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } } peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } }
I20260924 23:11:14.560254 20558 raft_consensus.cc:374] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:14.560298 20558 raft_consensus.cc:729] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 247c95b3bbae46d395cb9822ac4f9225, State: Initialized, Role: FOLLOWER
I20260924 23:11:14.560370 20558 consensus_queue.cc:260] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } } peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } }
I20260924 23:11:14.560559 20558 ts_tablet_manager.cc:1428] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:14.560941 20557 tablet_bootstrap.cc:654] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:14.561053 20561 tablet_bootstrap.cc:492] T 1e3c933b89c74fd68d67737c64aadd07 P 5816efe62c724db78e74db9e416e81ea: Bootstrap starting.
I20260924 23:11:14.561424 20321 heartbeater.cc:503] Master 127.12.212.61:41905 was elected leader, sending a full tablet report...
I20260924 23:11:14.561532 20561 tablet_bootstrap.cc:654] T 1e3c933b89c74fd68d67737c64aadd07 P 5816efe62c724db78e74db9e416e81ea: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:14.561558 20557 tablet_bootstrap.cc:492] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722: No bootstrap required, opened a new log
I20260924 23:11:14.561605 20557 ts_tablet_manager.cc:1397] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 23:11:14.561776 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.560729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.2:49989)
0924 23:11:14.560881 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.560921 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260924 23:11:14.561774 20557 raft_consensus.cc:348] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } } peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } }
I20260924 23:11:14.561849 20557 raft_consensus.cc:374] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:14.561870 20557 raft_consensus.cc:729] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0beb2bfd73a54e99a82ae77cfb709722, State: Initialized, Role: FOLLOWER
I20260924 23:11:14.561918 20557 consensus_queue.cc:260] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } } peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } }
I20260924 23:11:14.562050 20561 tablet_bootstrap.cc:492] T 1e3c933b89c74fd68d67737c64aadd07 P 5816efe62c724db78e74db9e416e81ea: No bootstrap required, opened a new log
I20260924 23:11:14.562068 20557 ts_tablet_manager.cc:1428] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:14.562103 20561 ts_tablet_manager.cc:1397] T 1e3c933b89c74fd68d67737c64aadd07 P 5816efe62c724db78e74db9e416e81ea: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:14.562259 20561 raft_consensus.cc:348] T 1e3c933b89c74fd68d67737c64aadd07 P 5816efe62c724db78e74db9e416e81ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } } peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } }
I20260924 23:11:14.562336 20561 raft_consensus.cc:374] T 1e3c933b89c74fd68d67737c64aadd07 P 5816efe62c724db78e74db9e416e81ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:14.562356 20561 raft_consensus.cc:729] T 1e3c933b89c74fd68d67737c64aadd07 P 5816efe62c724db78e74db9e416e81ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5816efe62c724db78e74db9e416e81ea, State: Initialized, Role: FOLLOWER
I20260924 23:11:14.562403 20561 consensus_queue.cc:260] T 1e3c933b89c74fd68d67737c64aadd07 P 5816efe62c724db78e74db9e416e81ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } } peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } }
I20260924 23:11:14.562534 20566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.562272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.3:38205)
0924 23:11:14.562459 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.562499 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":76,"threads_started":1}
I20260924 23:11:14.562558 20561 ts_tablet_manager.cc:1428] T 1e3c933b89c74fd68d67737c64aadd07 P 5816efe62c724db78e74db9e416e81ea: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:14.563009 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.562665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.2:47683)
0924 23:11:14.562922 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.562981 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":87,"threads_started":1}
I20260924 23:11:14.563277 20569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.562777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.1:41935)
0924 23:11:14.563206 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.563241 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":78,"threads_started":1}
I20260924 23:11:14.563297 20184 heartbeater.cc:503] Master 127.12.212.61:41905 was elected leader, sending a full tablet report...
I20260924 23:11:14.563656 20570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.563472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.1:43805)
0924 23:11:14.563583 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.563618 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":68,"threads_started":1}
I20260924 23:11:14.563845 20566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.563503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.3:59531)
0924 23:11:14.563791 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.563818 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":73,"threads_started":1}
I20260924 23:11:14.571709 20560 raft_consensus.cc:482] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:11:14.571810 20560 raft_consensus.cc:504] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } } peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } }
I20260924 23:11:14.571979 20560 leader_election.cc:288] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5816efe62c724db78e74db9e416e81ea (127.12.212.1:33293), 0beb2bfd73a54e99a82ae77cfb709722 (127.12.212.3:36715)
I20260924 23:11:14.572160 20409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e3c933b89c74fd68d67737c64aadd07" candidate_uuid: "247c95b3bbae46d395cb9822ac4f9225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0beb2bfd73a54e99a82ae77cfb709722" is_pre_election: true
I20260924 23:11:14.572213 20136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e3c933b89c74fd68d67737c64aadd07" candidate_uuid: "247c95b3bbae46d395cb9822ac4f9225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5816efe62c724db78e74db9e416e81ea" is_pre_election: true
I20260924 23:11:14.572261 20409 raft_consensus.cc:2454] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 247c95b3bbae46d395cb9822ac4f9225 in term 0.
I20260924 23:11:14.572290 20136 raft_consensus.cc:2454] T 1e3c933b89c74fd68d67737c64aadd07 P 5816efe62c724db78e74db9e416e81ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 247c95b3bbae46d395cb9822ac4f9225 in term 0.
I20260924 23:11:14.572410 20207 leader_election.cc:302] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0beb2bfd73a54e99a82ae77cfb709722, 247c95b3bbae46d395cb9822ac4f9225; no voters:
I20260924 23:11:14.572490 20560 raft_consensus.cc:2781] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:11:14.572527 20560 raft_consensus.cc:482] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:11:14.572548 20560 raft_consensus.cc:3037] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:14.573299 20560 raft_consensus.cc:504] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } } peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } }
I20260924 23:11:14.573418 20560 leader_election.cc:288] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [CANDIDATE]: Term 1 election: Requested vote from peers 5816efe62c724db78e74db9e416e81ea (127.12.212.1:33293), 0beb2bfd73a54e99a82ae77cfb709722 (127.12.212.3:36715)
I20260924 23:11:14.573618 20136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e3c933b89c74fd68d67737c64aadd07" candidate_uuid: "247c95b3bbae46d395cb9822ac4f9225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5816efe62c724db78e74db9e416e81ea"
I20260924 23:11:14.573702 20136 raft_consensus.cc:3037] T 1e3c933b89c74fd68d67737c64aadd07 P 5816efe62c724db78e74db9e416e81ea [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:14.573838 20409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e3c933b89c74fd68d67737c64aadd07" candidate_uuid: "247c95b3bbae46d395cb9822ac4f9225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0beb2bfd73a54e99a82ae77cfb709722"
I20260924 23:11:14.573915 20409 raft_consensus.cc:3037] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:14.574373 20136 raft_consensus.cc:2454] T 1e3c933b89c74fd68d67737c64aadd07 P 5816efe62c724db78e74db9e416e81ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 247c95b3bbae46d395cb9822ac4f9225 in term 1.
I20260924 23:11:14.574473 20409 raft_consensus.cc:2454] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 247c95b3bbae46d395cb9822ac4f9225 in term 1.
I20260924 23:11:14.574530 20208 leader_election.cc:302] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 247c95b3bbae46d395cb9822ac4f9225, 5816efe62c724db78e74db9e416e81ea; no voters:
I20260924 23:11:14.574743 20560 raft_consensus.cc:2781] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:14.574880 20560 raft_consensus.cc:686] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [term 1 LEADER]: Becoming Leader. State: Replica: 247c95b3bbae46d395cb9822ac4f9225, State: Running, Role: LEADER
I20260924 23:11:14.574930 20560 consensus_queue.cc:237] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } } peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } }
I20260924 23:11:14.575275 20568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.575140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.2:49967)
0924 23:11:14.575192 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.575237 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260924 23:11:14.575624 19817 catalog_manager.cc:5495] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 reported cstate change: term changed from 0 to 1, leader changed from <none> to 247c95b3bbae46d395cb9822ac4f9225 (127.12.212.2). New cstate: current_term: 1 leader_uuid: "247c95b3bbae46d395cb9822ac4f9225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247c95b3bbae46d395cb9822ac4f9225" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 38875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 } health_report { overall_health: UNKNOWN } } }
I20260924 23:11:14.587538 20554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.585134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56066 (local address 127.12.212.2:38875)
0924 23:11:14.585193 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:14.585195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:14.585203 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:11:14.585225 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:14.585225 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:14.585234 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:14.585275 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:11:14.585425 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:14.585926 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:14.586804 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:14.587003 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:14.587064 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:14.587241 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:11:14.587274 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:11:14.587325 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:14.587430 (+ 105us) server_negotiation.cc:299] Negotiation successful
0924 23:11:14.587454 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 23:11:14.588567 20136 raft_consensus.cc:1261] T 1e3c933b89c74fd68d67737c64aadd07 P 5816efe62c724db78e74db9e416e81ea [term 1 FOLLOWER]: Refusing update from remote peer 247c95b3bbae46d395cb9822ac4f9225: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:14.588577 20409 raft_consensus.cc:1261] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722 [term 1 FOLLOWER]: Refusing update from remote peer 247c95b3bbae46d395cb9822ac4f9225: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:14.588784 20560 consensus_queue.cc:1035] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5816efe62c724db78e74db9e416e81ea" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 33293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:14.588913 20572 consensus_queue.cc:1035] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0beb2bfd73a54e99a82ae77cfb709722" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:14.589648 20457 heartbeater.cc:503] Master 127.12.212.61:41905 was elected leader, sending a full tablet report...
I20260924 23:11:14.589744 20570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.589306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.1:43913)
0924 23:11:14.589670 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.589711 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337}
I20260924 23:11:14.589955 20571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.589796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.3:49053)
0924 23:11:14.589879 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.589918 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260924 23:11:14.590900 20568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.590655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.2:56767)
0924 23:11:14.590839 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.590870 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
I20260924 23:11:14.591068 20570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.590929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.1:42369)
0924 23:11:14.591017 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.591043 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260924 23:11:14.591508 20568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.591428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.2:60227)
0924 23:11:14.591463 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.591484 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260924 23:11:14.591794 20571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.591711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37579 (local address 127.12.212.3:59633)
0924 23:11:14.591745 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.591767 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260924 23:11:14.592315 19818 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:11:14.592393 19818 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:11:14.593683 19818 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfc0464cc67a40de8e97ffd2dfb0c48b: Sending DeleteTablet for 3 replicas of tablet 1e3c933b89c74fd68d67737c64aadd07
I20260924 23:11:14.593894 20116 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e3c933b89c74fd68d67737c64aadd07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:14 UTC) from {username='oryx', principal='oryx/127.12.212.61@KRBTEST.COM'} at 127.0.0.1:41012
I20260924 23:11:14.593963 20389 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e3c933b89c74fd68d67737c64aadd07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:14 UTC) from {username='oryx', principal='oryx/127.12.212.61@KRBTEST.COM'} at 127.0.0.1:53968
I20260924 23:11:14.594053 20253 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e3c933b89c74fd68d67737c64aadd07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:14 UTC) from {username='oryx', principal='oryx/127.12.212.61@KRBTEST.COM'} at 127.0.0.1:56062
I20260924 23:11:14.594152 20585 tablet_replica.cc:330] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722: stopping tablet replica
I20260924 23:11:14.594201 20585 raft_consensus.cc:2229] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:11:14.594199 20252 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e3c933b89c74fd68d67737c64aadd07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:14 UTC) from {username='oryx', principal='oryx/127.12.212.61@KRBTEST.COM'} at 127.0.0.1:56062
I20260924 23:11:14.594237 20585 raft_consensus.cc:2258] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:14.594213 20586 tablet_replica.cc:330] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225: stopping tablet replica
I20260924 23:11:14.594276 20586 raft_consensus.cc:2229] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:11:14.594332 20586 raft_consensus.cc:2258] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:14.594524 20585 ts_tablet_manager.cc:1905] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:14.594556 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 19965
I20260924 23:11:14.595252 20586 ts_tablet_manager.cc:1905] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:14.596815 20586 ts_tablet_manager.cc:1918] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:11:14.596874 20586 log.cc:1196] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/1e3c933b89c74fd68d67737c64aadd07
I20260924 23:11:14.597076 20586 ts_tablet_manager.cc:1939] T 1e3c933b89c74fd68d67737c64aadd07 P 247c95b3bbae46d395cb9822ac4f9225: Deleting consensus metadata
I20260924 23:11:14.597415 19803 catalog_manager.cc:4841] TS 247c95b3bbae46d395cb9822ac4f9225 (127.12.212.2:38875): tablet 1e3c933b89c74fd68d67737c64aadd07 (table test-table [id=614db433606b4aa19feb3c0a3d747d2a]) successfully deleted
W20260924 23:11:14.597627 19803 catalog_manager.cc:4808] TS 247c95b3bbae46d395cb9822ac4f9225 (127.12.212.2:38875): delete failed for tablet 1e3c933b89c74fd68d67737c64aadd07 because the tablet was not found. No further retry: Not found: Tablet not found: 1e3c933b89c74fd68d67737c64aadd07
I20260924 23:11:14.597872 20585 ts_tablet_manager.cc:1918] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:11:14.597934 20585 log.cc:1196] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/1e3c933b89c74fd68d67737c64aadd07
I20260924 23:11:14.598129 20585 ts_tablet_manager.cc:1939] T 1e3c933b89c74fd68d67737c64aadd07 P 0beb2bfd73a54e99a82ae77cfb709722: Deleting consensus metadata
I20260924 23:11:14.598726 19803 catalog_manager.cc:4841] TS 0beb2bfd73a54e99a82ae77cfb709722 (127.12.212.3:36715): tablet 1e3c933b89c74fd68d67737c64aadd07 (table test-table [id=614db433606b4aa19feb3c0a3d747d2a]) successfully deleted
I20260924 23:11:14.599893 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:14.599788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:33293 (local address 127.0.0.1:41026)
0924 23:11:14.599841 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:14.599865 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.1:33293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 23:11:14.600026 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 20190
W20260924 23:11:14.600026 19804 catalog_manager.cc:4584] TS 5816efe62c724db78e74db9e416e81ea (127.12.212.1:33293): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e3c933b89c74fd68d67737c64aadd07: Network error: Client connection negotiation failed: client connection to 127.12.212.1:33293: connect: Connection refused (error 111)
I20260924 23:11:14.605912 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 20326
I20260924 23:11:14.610674 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 19683
I20260924 23:11:14.614852 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 19872
2026-09-24T23:11:14Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6811 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:11:14 dist-test-slave-m0z6 krb5kdc[20589](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:11:14 dist-test-slave-m0z6 krb5kdc[20589](info): set up 2 sockets
Sep 24 23:11:14 dist-test-slave-m0z6 krb5kdc[20589](info): commencing operation
krb5kdc: starting...
W20260924 23:11:15.590900 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.955s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:11:15 dist-test-slave-m0z6 krb5kdc[20589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:11:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:11:15Z Disabled control of system clock
WARNING: no policy specified for oryx/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.62.keytab.
Entry for principal oryx/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.62.keytab.
I20260924 23:11:15.714901 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:35281
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.62.keytab
--principal=oryx/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:35281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:15.795073 20602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:15.795312 20602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:15.795343 20602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:15.796568 20602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:11:15.796620 20602 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 23:11:15.796638 20602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:15.796654 20602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:15.796670 20602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:11:15.796684 20602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:11:15.798295 20602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:35281
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.62.keytab
--principal=oryx/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:35281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:15.798560 20602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:15.798812 20602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:11:15.804030 20602 server_base.cc:1034] running on GCE node
W20260924 23:11:15.803998 20607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:15.803992 20610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:15.804003 20608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:15.804674 20602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:15.805004 20602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:15.806180 20602 hybrid_clock.cc:648] HybridClock initialized: now 1790291475806140 us; error 53 us; skew 500 ppm
Sep 24 23:11:15 dist-test-slave-m0z6 krb5kdc[20589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291475, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:15.808825 20602 init.cc:377] Logged in from keytab as oryx/127.12.212.62@KRBTEST.COM (short username oryx)
I20260924 23:11:15.809298 20602 webserver.cc:460] Webserver started at http://127.12.212.62:40487/ using document root <none> and password file <none>
I20260924 23:11:15.809513 20602 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:15.809563 20602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:15.809707 20602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:15.810891 20602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "0014010946794434a6979a1a749145cc"
format_stamp: "Formatted at 2026-09-24 23:11:15 on dist-test-slave-m0z6"
server_key: "1a6eef56977ccae51e90a79d42c94007"
server_key_iv: "05e0fed204e1f39b632cc6951cca7036"
server_key_version: "encryptionkey@0"
I20260924 23:11:15.811288 20602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "0014010946794434a6979a1a749145cc"
format_stamp: "Formatted at 2026-09-24 23:11:15 on dist-test-slave-m0z6"
server_key: "1a6eef56977ccae51e90a79d42c94007"
server_key_iv: "05e0fed204e1f39b632cc6951cca7036"
server_key_version: "encryptionkey@0"
I20260924 23:11:15.812753 20602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260924 23:11:15.813654 20617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:15.814028 20602 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:15.814131 20602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "0014010946794434a6979a1a749145cc"
format_stamp: "Formatted at 2026-09-24 23:11:15 on dist-test-slave-m0z6"
server_key: "1a6eef56977ccae51e90a79d42c94007"
server_key_iv: "05e0fed204e1f39b632cc6951cca7036"
server_key_version: "encryptionkey@0"
I20260924 23:11:15.814213 20602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:15.822032 20602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:15.824488 20602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:15.824642 20602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:15.829453 20602 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:35281
I20260924 23:11:15.829501 20679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:35281 every 8 connection(s)
I20260924 23:11:15.829856 20602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:11:15.830551 20680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:15.832834 20680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc: Bootstrap starting.
I20260924 23:11:15.833395 20680 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:15.833634 20680 log.cc:824] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:15.834313 20680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc: No bootstrap required, opened a new log
I20260924 23:11:15.836410 20680 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0014010946794434a6979a1a749145cc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 35281 } }
I20260924 23:11:15.836565 20680 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:15.836585 20680 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0014010946794434a6979a1a749145cc, State: Initialized, Role: FOLLOWER
I20260924 23:11:15.836704 20680 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0014010946794434a6979a1a749145cc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 35281 } }
I20260924 23:11:15.836778 20680 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:11:15.836802 20680 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:11:15.836834 20680 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:15.837360 20680 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0014010946794434a6979a1a749145cc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 35281 } }
I20260924 23:11:15.837447 20680 leader_election.cc:302] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0014010946794434a6979a1a749145cc; no voters:
I20260924 23:11:15.837592 20680 leader_election.cc:288] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:11:15.837669 20683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:15.837850 20683 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc [term 1 LEADER]: Becoming Leader. State: Replica: 0014010946794434a6979a1a749145cc, State: Running, Role: LEADER
I20260924 23:11:15.837853 20680 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:11:15.837968 20683 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0014010946794434a6979a1a749145cc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 35281 } }
I20260924 23:11:15.838376 20684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0014010946794434a6979a1a749145cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0014010946794434a6979a1a749145cc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 35281 } } }
I20260924 23:11:15.838426 20685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0014010946794434a6979a1a749145cc. Latest consensus state: current_term: 1 leader_uuid: "0014010946794434a6979a1a749145cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0014010946794434a6979a1a749145cc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 35281 } } }
I20260924 23:11:15.838469 20684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:15.838503 20685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:15.838655 20691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:11:15.839205 20691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:11:15.839962 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 20602
I20260924 23:11:15.840087 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:11:15.840309 13136 external_mini_cluster.cc:1428] Setting key 3044c57cbd56e0cf34ba8db768e36a2d
I20260924 23:11:15.841135 20691 catalog_manager.cc:1347] Generated new cluster ID: f8bac415de56488f894ecdb64eb5dbd6
I20260924 23:11:15.841181 20691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 23:11:15 dist-test-slave-m0z6 krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM
I20260924 23:11:15.858871 20691 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:11:15.859664 20691 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:11:15.862241 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:15.840820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57788 (local address 127.12.212.62:35281)
0924 23:11:15.841005 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:15.841016 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:15.850992 (+ 9976us) server_negotiation.cc:407] Connection header received
0924 23:11:15.851164 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:15.851172 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:15.851239 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:15.851405 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:15.852217 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:15.853678 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:15.854881 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:15.855549 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:15.855660 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:15.858794 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:15.858801 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:15.858801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:15.858818 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:15.860575 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:15.860976 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:15.860980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:15.860981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:15.861108 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:15.861317 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:15.861318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:15.861318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:15.861577 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:15.861725 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:15.861843 (+ 118us) server_negotiation.cc:299] Negotiation successful
0924 23:11:15.862010 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260924 23:11:15.865752 20691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc: Generated new TSK 0
I20260924 23:11:15.865973 20691 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.1.keytab.
Entry for principal oryx/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.1.keytab.
I20260924 23:11:15.899797 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:35281
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.1.keytab
--principal=oryx/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:16.002393 20706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:16.002652 20706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:16.002679 20706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:16.004045 20706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:16.004114 20706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:16.004137 20706 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 23:11:16.004159 20706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:11:16.005812 20706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.1.keytab
--principal=oryx/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.212.62:35281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:16.006130 20706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:16.006345 20706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:16.011798 20712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:16.011835 20711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:16.011927 20714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:16.011983 20706 server_base.cc:1034] running on GCE node
I20260924 23:11:16.012333 20706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:16.012604 20706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:16.013761 20706 hybrid_clock.cc:648] HybridClock initialized: now 1790291476013748 us; error 31 us; skew 500 ppm
Sep 24 23:11:16 dist-test-slave-m0z6 krb5kdc[20589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291476, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:16.016562 20706 init.cc:377] Logged in from keytab as oryx/127.12.212.1@KRBTEST.COM (short username oryx)
I20260924 23:11:16.017040 20706 webserver.cc:460] Webserver started at http://127.12.212.1:44803/ using document root <none> and password file <none>
I20260924 23:11:16.017305 20706 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:16.017354 20706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:16.017474 20706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:16.018599 20706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "da34cf54bf864b62a671126b0b163fe6"
format_stamp: "Formatted at 2026-09-24 23:11:16 on dist-test-slave-m0z6"
server_key: "8ed93c7a9ea9bea5936779d34684d137"
server_key_iv: "f2fdb8cf03f9a03a8b549f97e87d4a87"
server_key_version: "encryptionkey@0"
I20260924 23:11:16.018960 20706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "da34cf54bf864b62a671126b0b163fe6"
format_stamp: "Formatted at 2026-09-24 23:11:16 on dist-test-slave-m0z6"
server_key: "8ed93c7a9ea9bea5936779d34684d137"
server_key_iv: "f2fdb8cf03f9a03a8b549f97e87d4a87"
server_key_version: "encryptionkey@0"
I20260924 23:11:16.020324 20706 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:16.021198 20721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:16.021425 20706 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:16.021543 20706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "da34cf54bf864b62a671126b0b163fe6"
format_stamp: "Formatted at 2026-09-24 23:11:16 on dist-test-slave-m0z6"
server_key: "8ed93c7a9ea9bea5936779d34684d137"
server_key_iv: "f2fdb8cf03f9a03a8b549f97e87d4a87"
server_key_version: "encryptionkey@0"
I20260924 23:11:16.021608 20706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:16.036377 20706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:16.039538 20706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:16.039783 20706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:16.040479 20706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:16.040560 20706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:16.040598 20706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:16.040616 20706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:16.046587 20706 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:38081
I20260924 23:11:16.046859 20836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:38081 every 8 connection(s)
I20260924 23:11:16.047055 20706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
Sep 24 23:11:16 dist-test-slave-m0z6 krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291476, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM
I20260924 23:11:16.051352 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.040905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:57753 (local address 127.12.212.62:35281)
0924 23:11:16.040952 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.040955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.041192 (+ 237us) server_negotiation.cc:407] Connection header received
0924 23:11:16.041737 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.041738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.041753 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.041818 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:16.042773 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.043571 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.045075 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.045320 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.045384 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.048299 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:16.048301 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:16.048301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:16.048309 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:16.049697 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:16.050254 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:16.050255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:16.050256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:16.050318 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:16.050550 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:16.050550 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:16.050551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:16.050670 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:16.050781 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.051184 (+ 403us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.051279 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 23:11:16.051406 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.040920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:35281 (local address 127.12.212.1:57753)
0924 23:11:16.041073 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:16.041116 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:16.041616 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:16.041901 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:16.041906 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:16.042302 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:16.042612 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.042620 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.043671 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.043673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.044975 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.044979 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.045462 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.045463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.045645 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.046310 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:16.046331 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:16.048139 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:16.049811 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:16.049820 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:16.049822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:16.050163 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:16.050381 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:16.050381 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:16.050381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:16.050496 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:16.050770 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:16.050776 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:16.050918 (+ 142us) client_negotiation.cc:769] Sending connection context
0924 23:11:16.051058 (+ 140us) client_negotiation.cc:240] Negotiation successful
0924 23:11:16.051214 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
I20260924 23:11:16.051956 20837 heartbeater.cc:347] Connected to a master server at 127.12.212.62:35281
I20260924 23:11:16.052050 20837 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:16.052253 20837 heartbeater.cc:511] Master 127.12.212.62:35281 requested a full tablet report, sending...
I20260924 23:11:16.052870 20634 ts_manager.cc:194] Registered new tserver with Master: da34cf54bf864b62a671126b0b163fe6 (127.12.212.1:38081)
I20260924 23:11:16.053763 20634 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:57753
I20260924 23:11:16.056133 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 20706
I20260924 23:11:16.056243 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:11:16.056427 13136 external_mini_cluster.cc:1428] Setting key a4f31650b483948fb94d53f96caefb1d
I20260924 23:11:16.059227 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.053209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:53617 (local address 127.12.212.62:35281)
0924 23:11:16.053255 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.053257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.053324 (+ 67us) server_negotiation.cc:407] Connection header received
0924 23:11:16.053383 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.053384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.053392 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.053436 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:16.053791 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.054376 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.055184 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.055374 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.055436 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.056454 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:16.056455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:16.056456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:16.056471 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:16.057923 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:16.058355 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:16.058356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:16.058356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:16.058420 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:16.058617 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:16.058618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:16.058618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:16.058795 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:16.058858 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.059036 (+ 178us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.059124 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 23:11:16.059240 20843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.053122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:35281 (local address 127.12.212.1:53617)
0924 23:11:16.053267 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:16.053278 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:16.053338 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:16.053451 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:16.053452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:16.053644 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:16.053721 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.053723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.054469 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.054470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.055085 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.055087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.055441 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.055442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.055492 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.055925 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:16.055930 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:16.056334 (+ 404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:16.058032 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:16.058033 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:16.058034 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:16.058276 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:16.058483 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:16.058485 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:16.058485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:16.058556 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:16.058856 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:16.058857 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:16.058913 (+ 56us) client_negotiation.cc:769] Sending connection context
0924 23:11:16.058994 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 23:11:16.059124 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for oryx/127.12.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.2.keytab.
Entry for principal oryx/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.2.keytab.
I20260924 23:11:16.086735 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:35281
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.2.keytab
--principal=oryx/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:16.167495 20846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:16.167727 20846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:16.167754 20846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:16.169072 20846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:16.169126 20846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:16.169147 20846 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 23:11:16.169164 20846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:11:16.170810 20846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.2.keytab
--principal=oryx/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.212.62:35281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:16.171047 20846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:16.171270 20846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:16.176556 20854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:16.176584 20852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:16.176556 20851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:16.176720 20846 server_base.cc:1034] running on GCE node
I20260924 23:11:16.177013 20846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:16.177268 20846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:16.178511 20846 hybrid_clock.cc:648] HybridClock initialized: now 1790291476178487 us; error 42 us; skew 500 ppm
Sep 24 23:11:16 dist-test-slave-m0z6 krb5kdc[20589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291476, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:16.181188 20846 init.cc:377] Logged in from keytab as oryx/127.12.212.2@KRBTEST.COM (short username oryx)
I20260924 23:11:16.181737 20846 webserver.cc:460] Webserver started at http://127.12.212.2:41705/ using document root <none> and password file <none>
I20260924 23:11:16.181926 20846 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:16.181972 20846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:16.182085 20846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:16.183207 20846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "f7b176d2383842bb9e755d0f05cb9f0c"
format_stamp: "Formatted at 2026-09-24 23:11:16 on dist-test-slave-m0z6"
server_key: "e88382a1efc9f7f9826ec1b044a31b2f"
server_key_iv: "fbb62215f16a110a62824a4ca5cd7e72"
server_key_version: "encryptionkey@0"
I20260924 23:11:16.183528 20846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "f7b176d2383842bb9e755d0f05cb9f0c"
format_stamp: "Formatted at 2026-09-24 23:11:16 on dist-test-slave-m0z6"
server_key: "e88382a1efc9f7f9826ec1b044a31b2f"
server_key_iv: "fbb62215f16a110a62824a4ca5cd7e72"
server_key_version: "encryptionkey@0"
I20260924 23:11:16.184828 20846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:16.185709 20861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:16.185895 20846 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:16.186034 20846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "f7b176d2383842bb9e755d0f05cb9f0c"
format_stamp: "Formatted at 2026-09-24 23:11:16 on dist-test-slave-m0z6"
server_key: "e88382a1efc9f7f9826ec1b044a31b2f"
server_key_iv: "fbb62215f16a110a62824a4ca5cd7e72"
server_key_version: "encryptionkey@0"
I20260924 23:11:16.186100 20846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:16.195832 20846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:16.198284 20846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:16.198402 20846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:16.198983 20846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:16.199028 20846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:16.199051 20846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:16.199069 20846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 23:11:16 dist-test-slave-m0z6 krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291476, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM
I20260924 23:11:16.205842 20846 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:46595
I20260924 23:11:16.205924 20976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:46595 every 8 connection(s)
I20260924 23:11:16.206403 20846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:11:16.208671 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.199356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:58257 (local address 127.12.212.62:35281)
0924 23:11:16.199397 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.199399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.199629 (+ 230us) server_negotiation.cc:407] Connection header received
0924 23:11:16.200165 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.200166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.200180 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.200227 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:16.200973 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.201527 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.203123 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.203401 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.203467 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.205705 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:16.205706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:16.205707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:16.205716 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:16.207186 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:16.207627 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:16.207628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:16.207629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:16.207688 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:16.207922 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:16.207923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:16.207924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:16.208045 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:16.208114 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.208445 (+ 331us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.208535 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 23:11:16.208768 20870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.199354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:35281 (local address 127.12.212.2:58257)
0924 23:11:16.199500 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:16.199536 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:16.200031 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:16.200312 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:16.200316 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:16.200675 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:16.200874 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.200880 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.201613 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.201614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.203026 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.203029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.203493 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.203494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.203618 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.204166 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:16.204182 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:16.205587 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:16.207287 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:16.207292 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:16.207294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:16.207523 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:16.207755 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:16.207756 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:16.207756 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:16.207848 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:16.208160 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:16.208167 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:16.208300 (+ 133us) client_negotiation.cc:769] Sending connection context
0924 23:11:16.208392 (+ 92us) client_negotiation.cc:240] Negotiation successful
0924 23:11:16.208552 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":84,"threads_started":1}
I20260924 23:11:16.209372 20977 heartbeater.cc:347] Connected to a master server at 127.12.212.62:35281
I20260924 23:11:16.209461 20977 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:16.209674 20977 heartbeater.cc:511] Master 127.12.212.62:35281 requested a full tablet report, sending...
I20260924 23:11:16.210161 20633 ts_manager.cc:194] Registered new tserver with Master: f7b176d2383842bb9e755d0f05cb9f0c (127.12.212.2:46595)
I20260924 23:11:16.210636 20633 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:58257
I20260924 23:11:16.211838 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 20846
I20260924 23:11:16.211953 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:11:16.212177 13136 external_mini_cluster.cc:1428] Setting key c2a9a88bc5e3ddd3a844eb9a6e893105
I20260924 23:11:16.216392 20983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.210368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:35281 (local address 127.12.212.2:60583)
0924 23:11:16.210541 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:16.210550 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:16.210599 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:16.210830 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:16.210831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:16.211017 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:16.211089 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.211091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.211756 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.211757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.212542 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.212545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.212918 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.212919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.212987 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.213455 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:16.213460 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:16.213822 (+ 362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:16.215322 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:16.215323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:16.215323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:16.215491 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:16.215749 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:16.215749 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:16.215750 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:16.215810 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:16.216140 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:16.216141 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:16.216181 (+ 40us) client_negotiation.cc:769] Sending connection context
0924 23:11:16.216238 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 23:11:16.216322 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":76,"threads_started":1}
I20260924 23:11:16.216452 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.210474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:60583 (local address 127.12.212.62:35281)
0924 23:11:16.210525 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.210526 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.210589 (+ 63us) server_negotiation.cc:407] Connection header received
0924 23:11:16.210637 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.210637 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.210646 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.210725 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:16.211168 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.211645 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.212630 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.212838 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.212888 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.213970 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:16.213971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:16.213971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:16.213977 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:16.215142 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:16.215589 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:16.215590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:16.215591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:16.215657 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:16.215877 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:16.215878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:16.215878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:16.216063 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:16.216132 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.216284 (+ 152us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.216361 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
WARNING: no policy specified for oryx/127.12.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.3.keytab.
Entry for principal oryx/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.3.keytab.
I20260924 23:11:16.239969 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:35281
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.3.keytab
--principal=oryx/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:16.324210 20986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:16.324489 20986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:16.324518 20986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:16.325984 20986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:16.326050 20986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:16.326081 20986 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 23:11:16.326107 20986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:11:16.327903 20986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.3.keytab
--principal=oryx/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.212.62:35281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:16.328210 20986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:16.328451 20986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:16.333590 20992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:16.333583 20991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:16.333614 20986 server_base.cc:1034] running on GCE node
W20260924 23:11:16.333583 20994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:16.333981 20986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:16.334203 20986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:16.335355 20986 hybrid_clock.cc:648] HybridClock initialized: now 1790291476335339 us; error 28 us; skew 500 ppm
Sep 24 23:11:16 dist-test-slave-m0z6 krb5kdc[20589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291476, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:16.338157 20986 init.cc:377] Logged in from keytab as oryx/127.12.212.3@KRBTEST.COM (short username oryx)
I20260924 23:11:16.338771 20986 webserver.cc:460] Webserver started at http://127.12.212.3:45163/ using document root <none> and password file <none>
I20260924 23:11:16.339020 20986 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:16.339071 20986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:16.339166 20986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:16.340344 20986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "6bb78406254f4487b617881308a89902"
format_stamp: "Formatted at 2026-09-24 23:11:16 on dist-test-slave-m0z6"
server_key: "e9b5228006e4f5776d6b1b5d51aae9ec"
server_key_iv: "bc370a1c03b687aaf89a533bfeedbf01"
server_key_version: "encryptionkey@0"
I20260924 23:11:16.340814 20986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "6bb78406254f4487b617881308a89902"
format_stamp: "Formatted at 2026-09-24 23:11:16 on dist-test-slave-m0z6"
server_key: "e9b5228006e4f5776d6b1b5d51aae9ec"
server_key_iv: "bc370a1c03b687aaf89a533bfeedbf01"
server_key_version: "encryptionkey@0"
I20260924 23:11:16.342319 20986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:11:16.343289 21001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:16.343528 20986 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 23:11:16.343614 20986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "6bb78406254f4487b617881308a89902"
format_stamp: "Formatted at 2026-09-24 23:11:16 on dist-test-slave-m0z6"
server_key: "e9b5228006e4f5776d6b1b5d51aae9ec"
server_key_iv: "bc370a1c03b687aaf89a533bfeedbf01"
server_key_version: "encryptionkey@0"
I20260924 23:11:16.343683 20986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:16.369901 20986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:16.373239 20986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:16.373420 20986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:16.374104 20986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:16.374162 20986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:16.374194 20986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:16.374219 20986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 23:11:16 dist-test-slave-m0z6 krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291476, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM
I20260924 23:11:16.382089 20986 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:46471
I20260924 23:11:16.382195 21116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:46471 every 8 connection(s)
I20260924 23:11:16.382529 20986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:11:16.385540 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 20986
I20260924 23:11:16.385636 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:11:16.385818 13136 external_mini_cluster.cc:1428] Setting key c39f08aa2ccedf5d474131777b80c3c6
I20260924 23:11:16.386106 21010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.374428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:35281 (local address 127.12.212.3:60453)
0924 23:11:16.374619 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:16.374662 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:16.375364 (+ 702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:16.375688 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:16.375693 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:16.376223 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:16.376724 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.376734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.377794 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.377796 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.379216 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.379220 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.379801 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.379803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.379960 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.380597 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:16.380609 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:16.382453 (+ 1844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:16.384271 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:16.384282 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:16.384284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:16.384549 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:16.384774 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:16.384775 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:16.384776 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:16.384925 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:16.385333 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:16.385340 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:16.385538 (+ 198us) client_negotiation.cc:769] Sending connection context
0924 23:11:16.385657 (+ 119us) client_negotiation.cc:240] Negotiation successful
0924 23:11:16.385851 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260924 23:11:16.386457 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.374402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:60453 (local address 127.12.212.62:35281)
0924 23:11:16.374449 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.374451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.374768 (+ 317us) server_negotiation.cc:407] Connection header received
0924 23:11:16.375483 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.375484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.375498 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.375556 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:16.376844 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.377650 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.379329 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.379636 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.379706 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.382571 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:16.382573 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:16.382573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:16.382583 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:16.384183 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:16.384632 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:16.384633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:16.384634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:16.384703 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:16.385015 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:16.385017 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:16.385018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:16.385258 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:16.385328 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.386258 (+ 930us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.386358 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 23:11:16.389078 21117 heartbeater.cc:347] Connected to a master server at 127.12.212.62:35281
I20260924 23:11:16.389191 21117 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:16.389398 21117 heartbeater.cc:511] Master 127.12.212.62:35281 requested a full tablet report, sending...
I20260924 23:11:16.389812 20633 ts_manager.cc:194] Registered new tserver with Master: 6bb78406254f4487b617881308a89902 (127.12.212.3:46471)
I20260924 23:11:16.390341 20633 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:60453
I20260924 23:11:16.394865 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.388312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:35139 (local address 127.12.212.62:35281)
0924 23:11:16.388377 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.388378 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.388387 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:11:16.388410 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.388411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.388423 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.388488 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:16.388894 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.389608 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.390627 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.390940 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.391003 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.392140 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:16.392142 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:16.392142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:16.392150 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:16.393519 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:16.393991 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:16.393993 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:16.393993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:16.394064 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:16.394263 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:16.394264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:16.394265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:16.394390 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:16.394450 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.394631 (+ 181us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.394761 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 23:11:16.394882 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.388130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:35281 (local address 127.12.212.3:35139)
0924 23:11:16.388283 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:16.388295 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:16.388350 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:16.388497 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:16.388497 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:16.388718 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:16.388804 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.388806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.389730 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.389731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.390547 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.390550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.391061 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.391062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.391124 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.391656 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:16.391660 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:16.392023 (+ 363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:16.393611 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:16.393612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:16.393613 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:16.393815 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:16.394134 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:16.394135 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:16.394136 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:16.394207 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:16.394483 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:16.394484 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:16.394536 (+ 52us) client_negotiation.cc:769] Sending connection context
0924 23:11:16.394596 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 23:11:16.394761 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260924 23:11:16.397751 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 23:11:16 dist-test-slave-m0z6 krb5kdc[20589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291476, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 23:11:16 dist-test-slave-m0z6 krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:16 dist-test-slave-m0z6 krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
W20260924 23:11:16.421030 20703 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:16.415029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57798 (local address 127.12.212.62:35281)
0924 23:11:16.415089 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.415091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.415101 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:11:16.415130 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.415130 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.415140 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.415190 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:16.415770 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.416297 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.417304 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.417519 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.417598 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.420936 (+ 3338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57798: BlockingRecv error: recv got EOF from 127.0.0.1:57798 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 24 23:11:16 dist-test-slave-m0z6 krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291476, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM
I20260924 23:11:16.429584 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.422045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57808 (local address 127.12.212.62:35281)
0924 23:11:16.422124 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.422126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.422171 (+ 45us) server_negotiation.cc:407] Connection header received
0924 23:11:16.422244 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.422245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.422255 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.422311 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:16.422903 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.423465 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.424206 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.424406 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.424478 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.427033 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:16.427034 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:16.427035 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:16.427043 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:16.428528 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:16.428870 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:16.428872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:16.428873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:16.428925 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:16.429088 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:16.429089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:16.429089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:16.429207 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:16.429295 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.429402 (+ 107us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.429486 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 23:11:16.431425 20633 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:11:16.433086 20633 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:11:16.440994 21140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.436346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:46595 (local address 127.0.0.1:36878)
0924 23:11:16.436791 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:16.436799 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:16.436839 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:16.437207 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:16.437208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:16.437210 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:16.437241 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.437242 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.438475 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.438477 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.439744 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.439747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.440636 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.440637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.440722 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.440740 (+ 18us) client_negotiation.cc:769] Sending connection context
0924 23:11:16.440809 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 23:11:16.440869 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":24,"thread_start_us":71,"threads_started":1}
I20260924 23:11:16.441112 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.436253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59788 (local address 127.12.212.1:38081)
0924 23:11:16.436415 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.436420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.436735 (+ 315us) server_negotiation.cc:407] Connection header received
0924 23:11:16.436785 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.436788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.436806 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.436878 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:16.437235 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.438424 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.440139 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.440737 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.440817 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.440929 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.440970 (+ 41us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.441008 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260924 23:11:16.441416 21143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.436479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36878 (local address 127.12.212.2:46595)
0924 23:11:16.436622 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.436626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.436892 (+ 266us) server_negotiation.cc:407] Connection header received
0924 23:11:16.437066 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.437070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.437085 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.437177 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:16.437309 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.438380 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.439827 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.440541 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.440619 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.440699 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.441259 (+ 560us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.441307 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260924 23:11:16.441479 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.436114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:46471 (local address 127.0.0.1:60194)
0924 23:11:16.436654 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:16.436663 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:16.436702 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:16.437029 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:16.437032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:16.437035 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:16.437208 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.437210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.438444 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.438445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.439990 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.439993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.440958 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.440958 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.441010 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.441014 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:11:16.441055 (+ 41us) client_negotiation.cc:240] Negotiation successful
0924 23:11:16.441397 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":85,"thread_start_us":92,"threads_started":1}
I20260924 23:11:16.441598 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.436114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:38081 (local address 127.0.0.1:59788)
0924 23:11:16.436496 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:16.436559 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:16.436638 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:16.437032 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:16.437033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:16.437038 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:16.437151 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.437154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.438542 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.438543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.440072 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.440074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.440825 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.440826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.440901 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.440905 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:11:16.440943 (+ 38us) client_negotiation.cc:240] Negotiation successful
0924 23:11:16.440997 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":156,"thread_start_us":91,"threads_started":1}
I20260924 23:11:16.441753 21144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.436493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60194 (local address 127.12.212.3:46471)
0924 23:11:16.436780 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.436785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.436795 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:11:16.436840 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.436842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.436858 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.436938 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:16.437311 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.438321 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.440085 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.440787 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.440851 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.440928 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.441616 (+ 688us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.441666 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":82,"threads_started":1}
I20260924 23:11:16.442099 20911 tablet_service.cc:1467] Processing CreateTablet for tablet c978b5a2adc94be2a5dae56d028ff6bf (DEFAULT_TABLE table=test-table [id=07c7059bf617479b8994c7ed23be0652]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:16.442420 20911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c978b5a2adc94be2a5dae56d028ff6bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:16.442400 21051 tablet_service.cc:1467] Processing CreateTablet for tablet c978b5a2adc94be2a5dae56d028ff6bf (DEFAULT_TABLE table=test-table [id=07c7059bf617479b8994c7ed23be0652]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:16.442797 21051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c978b5a2adc94be2a5dae56d028ff6bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:16.443390 20757 tablet_service.cc:1467] Processing CreateTablet for tablet c978b5a2adc94be2a5dae56d028ff6bf (DEFAULT_TABLE table=test-table [id=07c7059bf617479b8994c7ed23be0652]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:16.443728 20757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c978b5a2adc94be2a5dae56d028ff6bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:16.445637 21145 tablet_bootstrap.cc:492] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c: Bootstrap starting.
I20260924 23:11:16.446259 21145 tablet_bootstrap.cc:654] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:16.446527 21145 log.cc:824] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:16.447494 21145 tablet_bootstrap.cc:492] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c: No bootstrap required, opened a new log
I20260924 23:11:16.447576 21145 ts_tablet_manager.cc:1397] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 23:11:16.448139 21148 tablet_bootstrap.cc:492] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6: Bootstrap starting.
I20260924 23:11:16.448210 21146 tablet_bootstrap.cc:492] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902: Bootstrap starting.
I20260924 23:11:16.448781 21146 tablet_bootstrap.cc:654] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:16.448815 21148 tablet_bootstrap.cc:654] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:16.449045 21146 log.cc:824] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:16.449126 21148 log.cc:824] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:16.449596 21145 raft_consensus.cc:348] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } } peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } }
I20260924 23:11:16.449771 21145 raft_consensus.cc:374] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:16.449801 21148 tablet_bootstrap.cc:492] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6: No bootstrap required, opened a new log
I20260924 23:11:16.449807 21146 tablet_bootstrap.cc:492] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902: No bootstrap required, opened a new log
I20260924 23:11:16.449806 21145 raft_consensus.cc:729] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7b176d2383842bb9e755d0f05cb9f0c, State: Initialized, Role: FOLLOWER
I20260924 23:11:16.449869 21148 ts_tablet_manager.cc:1397] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:11:16.449869 21146 ts_tablet_manager.cc:1397] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:11:16.450026 21145 consensus_queue.cc:260] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } } peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } }
I20260924 23:11:16.450309 21145 ts_tablet_manager.cc:1428] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 23:11:16.450342 20977 heartbeater.cc:503] Master 127.12.212.62:35281 was elected leader, sending a full tablet report...
I20260924 23:11:16.451670 21146 raft_consensus.cc:348] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } } peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } }
I20260924 23:11:16.451830 21146 raft_consensus.cc:374] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:16.451853 21146 raft_consensus.cc:729] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb78406254f4487b617881308a89902, State: Initialized, Role: FOLLOWER
I20260924 23:11:16.451953 21146 consensus_queue.cc:260] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } } peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } }
I20260924 23:11:16.451951 21148 raft_consensus.cc:348] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } } peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } }
I20260924 23:11:16.452119 21148 raft_consensus.cc:374] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:16.452160 21148 raft_consensus.cc:729] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da34cf54bf864b62a671126b0b163fe6, State: Initialized, Role: FOLLOWER
I20260924 23:11:16.452235 21117 heartbeater.cc:503] Master 127.12.212.62:35281 was elected leader, sending a full tablet report...
I20260924 23:11:16.452219 21146 ts_tablet_manager.cc:1428] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260924 23:11:16.452279 21148 consensus_queue.cc:260] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } } peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } }
I20260924 23:11:16.452564 21148 ts_tablet_manager.cc:1428] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:11:16.452600 20837 heartbeater.cc:503] Master 127.12.212.62:35281 was elected leader, sending a full tablet report...
W20260924 23:11:16.457015 20978 tablet.cc:2369] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:11:16.536612 21152 raft_consensus.cc:482] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:11:16.536746 21152 raft_consensus.cc:504] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } } peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } }
I20260924 23:11:16.537101 21152 leader_election.cc:288] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da34cf54bf864b62a671126b0b163fe6 (127.12.212.1:38081), f7b176d2383842bb9e755d0f05cb9f0c (127.12.212.2:46595)
I20260924 23:11:16.541289 21143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.537579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:49939 (local address 127.12.212.2:46595)
0924 23:11:16.537622 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.537623 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.537685 (+ 62us) server_negotiation.cc:407] Connection header received
0924 23:11:16.537745 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.537746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.537753 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.537785 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:16.538065 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.538786 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.540241 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.540834 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.540918 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.540926 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.541167 (+ 241us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.541210 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 23:11:16.541289 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.537361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:51095 (local address 127.12.212.1:38081)
0924 23:11:16.537418 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.537420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.537432 (+ 12us) server_negotiation.cc:407] Connection header received
0924 23:11:16.537582 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.537583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.537593 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.537669 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:16.537860 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.538643 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.540241 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.540840 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.540918 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.540926 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.541166 (+ 240us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.541209 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 23:11:16.541314 21154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.537474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:46595 (local address 127.12.212.3:49939)
0924 23:11:16.537634 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:16.537643 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:16.537691 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:16.537789 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:16.537790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:16.537791 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:16.537935 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.537937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.538841 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.538842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.540114 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.540118 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.540983 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.540984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.541072 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.541077 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 23:11:16.541151 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 23:11:16.541204 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":4,"thread_start_us":104,"threads_started":1}
I20260924 23:11:16.541317 21010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.537326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:38081 (local address 127.12.212.3:51095)
0924 23:11:16.537367 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:16.537379 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:16.537429 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:16.537686 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:16.537687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:16.537689 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:16.537762 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.537764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.538777 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.538778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.540114 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.540118 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.540951 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.540952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.541076 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.541078 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 23:11:16.541151 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 23:11:16.541204 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":38}
I20260924 23:11:16.541754 20931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c978b5a2adc94be2a5dae56d028ff6bf" candidate_uuid: "6bb78406254f4487b617881308a89902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" is_pre_election: true
I20260924 23:11:16.541862 20778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c978b5a2adc94be2a5dae56d028ff6bf" candidate_uuid: "6bb78406254f4487b617881308a89902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da34cf54bf864b62a671126b0b163fe6" is_pre_election: true
I20260924 23:11:16.541952 20931 raft_consensus.cc:2454] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bb78406254f4487b617881308a89902 in term 0.
I20260924 23:11:16.542006 20778 raft_consensus.cc:2454] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bb78406254f4487b617881308a89902 in term 0.
I20260924 23:11:16.542191 21005 leader_election.cc:302] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bb78406254f4487b617881308a89902, f7b176d2383842bb9e755d0f05cb9f0c; no voters:
I20260924 23:11:16.542326 21152 raft_consensus.cc:2781] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:11:16.542380 21152 raft_consensus.cc:482] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:11:16.542439 21152 raft_consensus.cc:3037] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:16.543521 21152 raft_consensus.cc:504] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } } peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } }
I20260924 23:11:16.543650 21152 leader_election.cc:288] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [CANDIDATE]: Term 1 election: Requested vote from peers da34cf54bf864b62a671126b0b163fe6 (127.12.212.1:38081), f7b176d2383842bb9e755d0f05cb9f0c (127.12.212.2:46595)
I20260924 23:11:16.543828 20778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c978b5a2adc94be2a5dae56d028ff6bf" candidate_uuid: "6bb78406254f4487b617881308a89902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da34cf54bf864b62a671126b0b163fe6"
I20260924 23:11:16.543865 20931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c978b5a2adc94be2a5dae56d028ff6bf" candidate_uuid: "6bb78406254f4487b617881308a89902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7b176d2383842bb9e755d0f05cb9f0c"
I20260924 23:11:16.543941 20931 raft_consensus.cc:3037] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:16.543929 20778 raft_consensus.cc:3037] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:16.544683 20778 raft_consensus.cc:2454] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bb78406254f4487b617881308a89902 in term 1.
I20260924 23:11:16.544682 20931 raft_consensus.cc:2454] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bb78406254f4487b617881308a89902 in term 1.
I20260924 23:11:16.544867 21005 leader_election.cc:302] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bb78406254f4487b617881308a89902, f7b176d2383842bb9e755d0f05cb9f0c; no voters:
I20260924 23:11:16.544971 21152 raft_consensus.cc:2781] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:16.545127 21152 raft_consensus.cc:686] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [term 1 LEADER]: Becoming Leader. State: Replica: 6bb78406254f4487b617881308a89902, State: Running, Role: LEADER
I20260924 23:11:16.545208 21152 consensus_queue.cc:237] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } } peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } }
I20260924 23:11:16.546269 20633 catalog_manager.cc:5495] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bb78406254f4487b617881308a89902 (127.12.212.3). New cstate: current_term: 1 leader_uuid: "6bb78406254f4487b617881308a89902" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } health_report { overall_health: UNKNOWN } } }
W20260924 23:11:16.547991 20838 tablet.cc:2369] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:11:16.560300 21144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.557947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60198 (local address 127.12.212.3:46471)
0924 23:11:16.558021 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.558023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.558033 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:11:16.558090 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.558091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.558109 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.558165 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:11:16.558298 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.558800 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.559577 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.559760 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.559811 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.559987 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:11:16.560091 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:11:16.560144 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.560212 (+ 68us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.560230 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260924 23:11:16.562204 20931 raft_consensus.cc:1261] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c [term 1 FOLLOWER]: Refusing update from remote peer 6bb78406254f4487b617881308a89902: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:16.562237 20778 raft_consensus.cc:1261] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6 [term 1 FOLLOWER]: Refusing update from remote peer 6bb78406254f4487b617881308a89902: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:16.562533 21152 consensus_queue.cc:1035] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:16.562702 21155 consensus_queue.cc:1035] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:16.565768 21160 mvcc.cc:205] Tried to move back new op lower bound from 7333033887996096512 to 7333033887930068992. Current Snapshot: MvccSnapshot[applied={T|T < 7333033887996096512}]
I20260924 23:11:16.565862 21159 mvcc.cc:205] Tried to move back new op lower bound from 7333033887996096512 to 7333033887930068992. Current Snapshot: MvccSnapshot[applied={T|T < 7333033887996096512}]
I20260924 23:11:16.567234 20634 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:11:16.567323 20634 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:11:16.568070 20634 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc: Sending DeleteTablet for 3 replicas of tablet c978b5a2adc94be2a5dae56d028ff6bf
I20260924 23:11:16.568375 21051 tablet_service.cc:1514] Processing DeleteTablet for tablet c978b5a2adc94be2a5dae56d028ff6bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:16 UTC) from {username='oryx', principal='oryx/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:60194
I20260924 23:11:16.568382 20757 tablet_service.cc:1514] Processing DeleteTablet for tablet c978b5a2adc94be2a5dae56d028ff6bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:16 UTC) from {username='oryx', principal='oryx/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:59788
I20260924 23:11:16.568567 20911 tablet_service.cc:1514] Processing DeleteTablet for tablet c978b5a2adc94be2a5dae56d028ff6bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:16 UTC) from {username='oryx', principal='oryx/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:36878
I20260924 23:11:16.568600 21167 tablet_replica.cc:330] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6: stopping tablet replica
I20260924 23:11:16.568696 21167 raft_consensus.cc:2229] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:11:16.568877 21167 raft_consensus.cc:2258] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:16.568904 20634 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:11:16.569052 20634 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:11:16.569052 21169 tablet_replica.cc:330] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c: stopping tablet replica
I20260924 23:11:16.569144 21169 raft_consensus.cc:2229] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:11:16.569494 21167 ts_tablet_manager.cc:1905] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:16.570361 21051 tablet_service.cc:1467] Processing CreateTablet for tablet bb22686c962e43d597c08c2719f0194d (DEFAULT_TABLE table=test-table [id=7cca3dc1dc40475696d63859978f18d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:16.570385 20757 tablet_service.cc:1467] Processing CreateTablet for tablet bb22686c962e43d597c08c2719f0194d (DEFAULT_TABLE table=test-table [id=7cca3dc1dc40475696d63859978f18d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:16.570487 21051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb22686c962e43d597c08c2719f0194d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:16.570513 20757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb22686c962e43d597c08c2719f0194d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:16.571375 21167 ts_tablet_manager.cc:1918] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:11:16.571451 21167 log.cc:1196] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/wal/wals/c978b5a2adc94be2a5dae56d028ff6bf
I20260924 23:11:16.571672 21167 ts_tablet_manager.cc:1939] T c978b5a2adc94be2a5dae56d028ff6bf P da34cf54bf864b62a671126b0b163fe6: Deleting consensus metadata
I20260924 23:11:16.571851 21148 tablet_bootstrap.cc:492] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6: Bootstrap starting.
I20260924 23:11:16.571851 21146 tablet_bootstrap.cc:492] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902: Bootstrap starting.
I20260924 23:11:16.572216 21146 tablet_bootstrap.cc:654] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:16.572240 21148 tablet_bootstrap.cc:654] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:16.572794 21146 tablet_bootstrap.cc:492] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902: No bootstrap required, opened a new log
I20260924 23:11:16.572842 21146 ts_tablet_manager.cc:1397] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 23:11:16.572980 21146 raft_consensus.cc:348] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.573033 21146 raft_consensus.cc:374] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:16.573045 21146 raft_consensus.cc:729] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb78406254f4487b617881308a89902, State: Initialized, Role: FOLLOWER
I20260924 23:11:16.573079 21146 consensus_queue.cc:260] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.573156 21146 ts_tablet_manager.cc:1428] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:16.573151 20621 catalog_manager.cc:4841] TS da34cf54bf864b62a671126b0b163fe6 (127.12.212.1:38081): tablet c978b5a2adc94be2a5dae56d028ff6bf (table test-table [id=07c7059bf617479b8994c7ed23be0652]) successfully deleted
I20260924 23:11:16.573314 20911 tablet_service.cc:1467] Processing CreateTablet for tablet bb22686c962e43d597c08c2719f0194d (DEFAULT_TABLE table=test-table [id=7cca3dc1dc40475696d63859978f18d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:16.573428 20911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb22686c962e43d597c08c2719f0194d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:16.574815 21145 tablet_bootstrap.cc:492] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c: Bootstrap starting.
I20260924 23:11:16.575220 21145 tablet_bootstrap.cc:654] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:16.575687 21155 raft_consensus.cc:482] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:11:16.575747 21155 raft_consensus.cc:504] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.575870 21155 leader_election.cc:288] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da34cf54bf864b62a671126b0b163fe6 (127.12.212.1:38081), f7b176d2383842bb9e755d0f05cb9f0c (127.12.212.2:46595)
I20260924 23:11:16.576010 20778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb22686c962e43d597c08c2719f0194d" candidate_uuid: "6bb78406254f4487b617881308a89902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da34cf54bf864b62a671126b0b163fe6" is_pre_election: true
I20260924 23:11:16.576076 20931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb22686c962e43d597c08c2719f0194d" candidate_uuid: "6bb78406254f4487b617881308a89902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" is_pre_election: true
I20260924 23:11:16.576242 21145 tablet_bootstrap.cc:492] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c: No bootstrap required, opened a new log
I20260924 23:11:16.576294 21145 ts_tablet_manager.cc:1397] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
W20260924 23:11:16.576323 21005 leader_election.cc:341] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da34cf54bf864b62a671126b0b163fe6 (127.12.212.1:38081): Illegal state: must be running to vote when last-logged opid is not known
W20260924 23:11:16.576421 21005 leader_election.cc:341] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f7b176d2383842bb9e755d0f05cb9f0c (127.12.212.2:46595): Illegal state: must be running to vote when last-logged opid is not known
I20260924 23:11:16.576453 21005 leader_election.cc:302] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb78406254f4487b617881308a89902; no voters: da34cf54bf864b62a671126b0b163fe6, f7b176d2383842bb9e755d0f05cb9f0c
I20260924 23:11:16.576439 21145 raft_consensus.cc:348] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.576498 21145 raft_consensus.cc:374] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:16.576525 21145 raft_consensus.cc:729] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7b176d2383842bb9e755d0f05cb9f0c, State: Initialized, Role: FOLLOWER
I20260924 23:11:16.576565 21145 consensus_queue.cc:260] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.576673 21145 ts_tablet_manager.cc:1428] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 23:11:16.576848 21155 raft_consensus.cc:2726] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 23:11:16.577090 21148 tablet_bootstrap.cc:492] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6: No bootstrap required, opened a new log
I20260924 23:11:16.577143 21148 ts_tablet_manager.cc:1397] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260924 23:11:16.577271 21148 raft_consensus.cc:348] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.577340 21148 raft_consensus.cc:374] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:16.577365 21148 raft_consensus.cc:729] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da34cf54bf864b62a671126b0b163fe6, State: Initialized, Role: FOLLOWER
I20260924 23:11:16.577411 21148 consensus_queue.cc:260] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.577517 21148 ts_tablet_manager.cc:1428] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:16.575877 21168 tablet_replica.cc:330] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902: stopping tablet replica
I20260924 23:11:16.578263 21168 raft_consensus.cc:2229] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:11:16.578516 21168 raft_consensus.cc:2258] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:16.578735 21169 raft_consensus.cc:2258] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:16.579216 21168 ts_tablet_manager.cc:1905] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:16.580814 21168 ts_tablet_manager.cc:1918] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:11:16.580921 21168 log.cc:1196] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/c978b5a2adc94be2a5dae56d028ff6bf
I20260924 23:11:16.581014 21169 ts_tablet_manager.cc:1905] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:16.581193 21168 ts_tablet_manager.cc:1939] T c978b5a2adc94be2a5dae56d028ff6bf P 6bb78406254f4487b617881308a89902: Deleting consensus metadata
I20260924 23:11:16.581571 20619 catalog_manager.cc:4841] TS 6bb78406254f4487b617881308a89902 (127.12.212.3:46471): tablet c978b5a2adc94be2a5dae56d028ff6bf (table test-table [id=07c7059bf617479b8994c7ed23be0652]) successfully deleted
I20260924 23:11:16.582448 21169 ts_tablet_manager.cc:1918] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:11:16.582517 21169 log.cc:1196] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/c978b5a2adc94be2a5dae56d028ff6bf
I20260924 23:11:16.582788 21169 ts_tablet_manager.cc:1939] T c978b5a2adc94be2a5dae56d028ff6bf P f7b176d2383842bb9e755d0f05cb9f0c: Deleting consensus metadata
I20260924 23:11:16.583194 20621 catalog_manager.cc:4841] TS f7b176d2383842bb9e755d0f05cb9f0c (127.12.212.2:46595): tablet c978b5a2adc94be2a5dae56d028ff6bf (table test-table [id=07c7059bf617479b8994c7ed23be0652]) successfully deleted
I20260924 23:11:16.612803 21153 raft_consensus.cc:482] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:11:16.612897 21153 raft_consensus.cc:504] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.613319 21153 leader_election.cc:288] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7b176d2383842bb9e755d0f05cb9f0c (127.12.212.2:46595), 6bb78406254f4487b617881308a89902 (127.12.212.3:46471)
I20260924 23:11:16.616523 21144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.613727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:36079 (local address 127.12.212.3:46471)
0924 23:11:16.613795 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.613797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.613875 (+ 78us) server_negotiation.cc:407] Connection header received
0924 23:11:16.613956 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.613957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.613966 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.614009 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:16.614161 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.614858 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.615791 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.616195 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.616264 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.616272 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.616421 (+ 149us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.616452 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 23:11:16.616540 21174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.613515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:46471 (local address 127.12.212.1:36079)
0924 23:11:16.613828 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:16.613836 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:16.613907 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:16.614056 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:16.614056 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:16.614058 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:16.614107 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.614108 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.614947 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.614948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.615713 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.615715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.616282 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.616283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.616357 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.616362 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 23:11:16.616412 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 23:11:16.616452 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":131,"threads_started":1}
I20260924 23:11:16.617012 21173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.613492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:46595 (local address 127.12.212.1:46229)
0924 23:11:16.613616 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:16.613631 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:16.613711 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:16.613868 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:16.613869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:16.613872 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:16.613949 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.613952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.614849 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.614849 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.616085 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.616088 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.616770 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.616771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.616844 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.616847 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:11:16.616895 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 23:11:16.616934 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260924 23:11:16.617010 21143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.613549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:46229 (local address 127.12.212.2:46595)
0924 23:11:16.613617 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.613619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.613683 (+ 64us) server_negotiation.cc:407] Connection header received
0924 23:11:16.613792 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.613793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.613803 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.613857 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:16.614026 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.614777 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.616172 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.616678 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.616744 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.616752 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.616911 (+ 159us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.616950 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 23:11:16.617174 21071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb22686c962e43d597c08c2719f0194d" candidate_uuid: "da34cf54bf864b62a671126b0b163fe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bb78406254f4487b617881308a89902" is_pre_election: true
I20260924 23:11:16.617269 20931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb22686c962e43d597c08c2719f0194d" candidate_uuid: "da34cf54bf864b62a671126b0b163fe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" is_pre_election: true
I20260924 23:11:16.617303 21071 raft_consensus.cc:2454] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da34cf54bf864b62a671126b0b163fe6 in term 0.
I20260924 23:11:16.617343 20931 raft_consensus.cc:2454] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da34cf54bf864b62a671126b0b163fe6 in term 0.
I20260924 23:11:16.617501 20725 leader_election.cc:302] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da34cf54bf864b62a671126b0b163fe6, f7b176d2383842bb9e755d0f05cb9f0c; no voters:
I20260924 23:11:16.617676 21153 raft_consensus.cc:2781] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:11:16.617709 21153 raft_consensus.cc:482] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:11:16.617727 21153 raft_consensus.cc:3037] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:16.618413 21153 raft_consensus.cc:504] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.618533 21153 leader_election.cc:288] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [CANDIDATE]: Term 1 election: Requested vote from peers f7b176d2383842bb9e755d0f05cb9f0c (127.12.212.2:46595), 6bb78406254f4487b617881308a89902 (127.12.212.3:46471)
I20260924 23:11:16.618682 21071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb22686c962e43d597c08c2719f0194d" candidate_uuid: "da34cf54bf864b62a671126b0b163fe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bb78406254f4487b617881308a89902"
I20260924 23:11:16.618721 20931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb22686c962e43d597c08c2719f0194d" candidate_uuid: "da34cf54bf864b62a671126b0b163fe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7b176d2383842bb9e755d0f05cb9f0c"
I20260924 23:11:16.618791 20931 raft_consensus.cc:3037] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:16.618793 21071 raft_consensus.cc:3037] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:16.619365 20931 raft_consensus.cc:2454] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da34cf54bf864b62a671126b0b163fe6 in term 1.
I20260924 23:11:16.619514 21071 raft_consensus.cc:2454] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da34cf54bf864b62a671126b0b163fe6 in term 1.
I20260924 23:11:16.619518 20725 leader_election.cc:302] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da34cf54bf864b62a671126b0b163fe6, f7b176d2383842bb9e755d0f05cb9f0c; no voters:
I20260924 23:11:16.619623 21153 raft_consensus.cc:2781] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:16.619753 21153 raft_consensus.cc:686] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [term 1 LEADER]: Becoming Leader. State: Replica: da34cf54bf864b62a671126b0b163fe6, State: Running, Role: LEADER
I20260924 23:11:16.619820 21153 consensus_queue.cc:237] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.620476 20634 catalog_manager.cc:5495] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 reported cstate change: term changed from 0 to 1, leader changed from <none> to da34cf54bf864b62a671126b0b163fe6 (127.12.212.1). New cstate: current_term: 1 leader_uuid: "da34cf54bf864b62a671126b0b163fe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } health_report { overall_health: UNKNOWN } } }
W20260924 23:11:16.633206 21118 tablet.cc:2369] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 24 23:11:16 dist-test-slave-m0z6 krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291475, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.62@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM
I20260924 23:11:16.663259 21182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.655494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:35281 (local address 127.0.0.1:57814)
0924 23:11:16.655666 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:16.655677 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:16.655736 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:16.655935 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:16.655936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:16.656254 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:16.656308 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.656310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.657238 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.657239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.657789 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.657791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.658293 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.658294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.658344 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.658966 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:16.658971 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:16.660288 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:16.662069 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:16.662077 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:16.662079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:16.662288 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:16.662507 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:16.662508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:16.662508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:16.662571 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:16.662931 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:16.662938 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:16.662989 (+ 51us) client_negotiation.cc:769] Sending connection context
0924 23:11:16.663061 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 23:11:16.663171 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
I20260924 23:11:16.663357 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.655572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57814 (local address 127.12.212.62:35281)
0924 23:11:16.655645 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.655648 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.655735 (+ 87us) server_negotiation.cc:407] Connection header received
0924 23:11:16.655783 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.655785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.655797 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.655878 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:16.656392 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.657139 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.657930 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.658199 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.658276 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.660387 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:16.660390 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:16.660390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:16.660398 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:16.661972 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:16.662361 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:16.662363 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:16.662363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:16.662436 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:16.662626 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:16.662626 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:16.662627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:16.662806 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:16.662886 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.663096 (+ 210us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.663238 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":38}
I20260924 23:11:16.665786 20634 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:57814:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 23:11:16.666050 20634 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:11:16.667644 21051 tablet_service.cc:1467] Processing CreateTablet for tablet cb3ee737d92549409d1221ce98ba7c41 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7a74fcc9950947419b94c23a24a32b06]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 23:11:16.667676 20757 tablet_service.cc:1467] Processing CreateTablet for tablet cb3ee737d92549409d1221ce98ba7c41 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7a74fcc9950947419b94c23a24a32b06]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 23:11:16.667825 21051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb3ee737d92549409d1221ce98ba7c41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:16.667832 20757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb3ee737d92549409d1221ce98ba7c41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:16.669001 21146 tablet_bootstrap.cc:492] T cb3ee737d92549409d1221ce98ba7c41 P 6bb78406254f4487b617881308a89902: Bootstrap starting.
I20260924 23:11:16.669132 20911 tablet_service.cc:1467] Processing CreateTablet for tablet cb3ee737d92549409d1221ce98ba7c41 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7a74fcc9950947419b94c23a24a32b06]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 23:11:16.669274 20911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb3ee737d92549409d1221ce98ba7c41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:16.669442 21146 tablet_bootstrap.cc:654] T cb3ee737d92549409d1221ce98ba7c41 P 6bb78406254f4487b617881308a89902: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:16.670048 21146 tablet_bootstrap.cc:492] T cb3ee737d92549409d1221ce98ba7c41 P 6bb78406254f4487b617881308a89902: No bootstrap required, opened a new log
I20260924 23:11:16.670095 21146 ts_tablet_manager.cc:1397] T cb3ee737d92549409d1221ce98ba7c41 P 6bb78406254f4487b617881308a89902: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 23:11:16.670264 21146 raft_consensus.cc:348] T cb3ee737d92549409d1221ce98ba7c41 P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.670333 21146 raft_consensus.cc:374] T cb3ee737d92549409d1221ce98ba7c41 P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:16.670352 21146 raft_consensus.cc:729] T cb3ee737d92549409d1221ce98ba7c41 P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb78406254f4487b617881308a89902, State: Initialized, Role: FOLLOWER
I20260924 23:11:16.670394 21146 consensus_queue.cc:260] T cb3ee737d92549409d1221ce98ba7c41 P 6bb78406254f4487b617881308a89902 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.670483 21146 ts_tablet_manager.cc:1428] T cb3ee737d92549409d1221ce98ba7c41 P 6bb78406254f4487b617881308a89902: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:16.670918 21155 tablet_replica.cc:439] T cb3ee737d92549409d1221ce98ba7c41 P 6bb78406254f4487b617881308a89902: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } } }
I20260924 23:11:16.671017 21155 tablet_replica.cc:442] T cb3ee737d92549409d1221ce98ba7c41 P 6bb78406254f4487b617881308a89902: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:11:16.671108 21145 tablet_bootstrap.cc:492] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c: Bootstrap starting.
I20260924 23:11:16.671499 21145 tablet_bootstrap.cc:654] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:16.672264 21145 tablet_bootstrap.cc:492] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c: No bootstrap required, opened a new log
I20260924 23:11:16.672317 21145 ts_tablet_manager.cc:1397] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 23:11:16.672498 21145 raft_consensus.cc:348] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.672559 21145 raft_consensus.cc:374] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:16.672575 21145 raft_consensus.cc:729] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7b176d2383842bb9e755d0f05cb9f0c, State: Initialized, Role: FOLLOWER
I20260924 23:11:16.672591 21148 tablet_bootstrap.cc:492] T cb3ee737d92549409d1221ce98ba7c41 P da34cf54bf864b62a671126b0b163fe6: Bootstrap starting.
I20260924 23:11:16.672616 21145 consensus_queue.cc:260] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.672703 21145 ts_tablet_manager.cc:1428] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 23:11:16.673007 21148 tablet_bootstrap.cc:654] T cb3ee737d92549409d1221ce98ba7c41 P da34cf54bf864b62a671126b0b163fe6: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:16.673471 21151 tablet_replica.cc:439] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } } }
I20260924 23:11:16.673588 21151 tablet_replica.cc:442] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:11:16.673911 21148 tablet_bootstrap.cc:492] T cb3ee737d92549409d1221ce98ba7c41 P da34cf54bf864b62a671126b0b163fe6: No bootstrap required, opened a new log
I20260924 23:11:16.673969 21148 ts_tablet_manager.cc:1397] T cb3ee737d92549409d1221ce98ba7c41 P da34cf54bf864b62a671126b0b163fe6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:16.674113 21148 raft_consensus.cc:348] T cb3ee737d92549409d1221ce98ba7c41 P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.674188 21148 raft_consensus.cc:374] T cb3ee737d92549409d1221ce98ba7c41 P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:16.674214 21148 raft_consensus.cc:729] T cb3ee737d92549409d1221ce98ba7c41 P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da34cf54bf864b62a671126b0b163fe6, State: Initialized, Role: FOLLOWER
I20260924 23:11:16.674260 21148 consensus_queue.cc:260] T cb3ee737d92549409d1221ce98ba7c41 P da34cf54bf864b62a671126b0b163fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.674360 21148 ts_tablet_manager.cc:1428] T cb3ee737d92549409d1221ce98ba7c41 P da34cf54bf864b62a671126b0b163fe6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:16.674428 21153 tablet_replica.cc:439] T cb3ee737d92549409d1221ce98ba7c41 P da34cf54bf864b62a671126b0b163fe6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } } }
I20260924 23:11:16.674523 21153 tablet_replica.cc:442] T cb3ee737d92549409d1221ce98ba7c41 P da34cf54bf864b62a671126b0b163fe6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:11:16.761298 21151 raft_consensus.cc:482] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:11:16.761392 21151 raft_consensus.cc:504] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.761701 21151 leader_election.cc:288] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da34cf54bf864b62a671126b0b163fe6 (127.12.212.1:38081), 6bb78406254f4487b617881308a89902 (127.12.212.3:46471)
I20260924 23:11:16.764634 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.761987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:46451 (local address 127.12.212.1:38081)
0924 23:11:16.762026 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.762028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.762070 (+ 42us) server_negotiation.cc:407] Connection header received
0924 23:11:16.762144 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.762145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.762155 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.762206 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:16.762340 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.762969 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.763984 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.764333 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.764399 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.764406 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.764521 (+ 115us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.764554 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260924 23:11:16.764629 21144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.761947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:43471 (local address 127.12.212.3:46471)
0924 23:11:16.762020 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.762023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.762134 (+ 111us) server_negotiation.cc:407] Connection header received
0924 23:11:16.762193 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.762194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.762203 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.762247 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:16.762399 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.762952 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.763935 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.764309 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.764362 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.764370 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.764518 (+ 148us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.764562 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 23:11:16.764654 21187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.761893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:46471 (local address 127.12.212.2:43471)
0924 23:11:16.762048 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:16.762058 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:16.762129 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:16.762263 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:16.762264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:16.762267 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:16.762330 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.762332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.763028 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.763029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.763839 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.763841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.764381 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.764382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.764451 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.764455 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:11:16.764498 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 23:11:16.764547 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":3,"thread_start_us":66,"threads_started":1}
I20260924 23:11:16.764657 21186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.761892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:38081 (local address 127.12.212.2:46451)
0924 23:11:16.762015 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:16.762026 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:16.762085 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:16.762219 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:16.762220 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:16.762223 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:16.762283 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.762285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.763056 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.763058 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.763843 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.763845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.764411 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.764412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.764457 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.764461 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:11:16.764506 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 23:11:16.764548 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":40,"thread_start_us":67,"threads_started":1}
I20260924 23:11:16.764879 21071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb3ee737d92549409d1221ce98ba7c41" candidate_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bb78406254f4487b617881308a89902" is_pre_election: true
I20260924 23:11:16.764977 21071 raft_consensus.cc:2454] T cb3ee737d92549409d1221ce98ba7c41 P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7b176d2383842bb9e755d0f05cb9f0c in term 0.
I20260924 23:11:16.764962 20778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb3ee737d92549409d1221ce98ba7c41" candidate_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da34cf54bf864b62a671126b0b163fe6" is_pre_election: true
I20260924 23:11:16.765050 20778 raft_consensus.cc:2454] T cb3ee737d92549409d1221ce98ba7c41 P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7b176d2383842bb9e755d0f05cb9f0c in term 0.
I20260924 23:11:16.765139 20863 leader_election.cc:302] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bb78406254f4487b617881308a89902, f7b176d2383842bb9e755d0f05cb9f0c; no voters:
I20260924 23:11:16.765280 21151 raft_consensus.cc:2781] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:11:16.765326 21151 raft_consensus.cc:482] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:11:16.765347 21151 raft_consensus.cc:3037] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:16.766021 21151 raft_consensus.cc:504] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.766131 21151 leader_election.cc:288] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c [CANDIDATE]: Term 1 election: Requested vote from peers da34cf54bf864b62a671126b0b163fe6 (127.12.212.1:38081), 6bb78406254f4487b617881308a89902 (127.12.212.3:46471)
I20260924 23:11:16.766299 21071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb3ee737d92549409d1221ce98ba7c41" candidate_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bb78406254f4487b617881308a89902"
I20260924 23:11:16.766372 21071 raft_consensus.cc:3037] T cb3ee737d92549409d1221ce98ba7c41 P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:16.766359 20778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb3ee737d92549409d1221ce98ba7c41" candidate_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da34cf54bf864b62a671126b0b163fe6"
I20260924 23:11:16.766441 20778 raft_consensus.cc:3037] T cb3ee737d92549409d1221ce98ba7c41 P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:16.767019 21071 raft_consensus.cc:2454] T cb3ee737d92549409d1221ce98ba7c41 P 6bb78406254f4487b617881308a89902 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7b176d2383842bb9e755d0f05cb9f0c in term 1.
I20260924 23:11:16.767119 20778 raft_consensus.cc:2454] T cb3ee737d92549409d1221ce98ba7c41 P da34cf54bf864b62a671126b0b163fe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7b176d2383842bb9e755d0f05cb9f0c in term 1.
I20260924 23:11:16.767210 20863 leader_election.cc:302] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bb78406254f4487b617881308a89902, f7b176d2383842bb9e755d0f05cb9f0c; no voters:
I20260924 23:11:16.767297 21151 raft_consensus.cc:2781] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:16.767416 21151 raft_consensus.cc:686] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c [term 1 LEADER]: Becoming Leader. State: Replica: f7b176d2383842bb9e755d0f05cb9f0c, State: Running, Role: LEADER
I20260924 23:11:16.767475 21151 consensus_queue.cc:237] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.767791 21188 tablet_replica.cc:439] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c: TxnStatusTablet state changed. Reason: New leader f7b176d2383842bb9e755d0f05cb9f0c. Latest consensus state: current_term: 1 leader_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } } }
I20260924 23:11:16.767889 21188 tablet_replica.cc:442] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c: This TxnStatusTablet replica's current role is: LEADER
I20260924 23:11:16.768105 21190 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 23:11:16.768203 21190 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 23:11:16.768151 20634 catalog_manager.cc:5495] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c reported cstate change: term changed from 0 to 1, leader changed from <none> to f7b176d2383842bb9e755d0f05cb9f0c (127.12.212.2). New cstate: current_term: 1 leader_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } health_report { overall_health: UNKNOWN } } }
I20260924 23:11:16.789443 21143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.786502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36884 (local address 127.12.212.2:46595)
0924 23:11:16.786557 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.786559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.786569 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:11:16.786659 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.786660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.786677 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.786767 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:11:16.786918 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.787458 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.788584 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.788828 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.788880 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.789118 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:11:16.789223 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:11:16.789284 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.789341 (+ 57us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.789361 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 23:11:16.789469 21182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.786432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:46595 (local address 127.0.0.1:36884)
0924 23:11:16.786504 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:16.786517 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:16.786577 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:16.786775 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:16.786776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:16.786777 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 23:11:16.786846 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.786849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.787524 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.787525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.788499 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.788501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.788891 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.788892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.788976 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.788996 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 23:11:16.789282 (+ 286us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 23:11:16.789287 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 23:11:16.789337 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 23:11:16.789383 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 23:11:16.790745 20778 raft_consensus.cc:1261] T cb3ee737d92549409d1221ce98ba7c41 P da34cf54bf864b62a671126b0b163fe6 [term 1 FOLLOWER]: Refusing update from remote peer f7b176d2383842bb9e755d0f05cb9f0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:16.790861 21071 raft_consensus.cc:1261] T cb3ee737d92549409d1221ce98ba7c41 P 6bb78406254f4487b617881308a89902 [term 1 FOLLOWER]: Refusing update from remote peer f7b176d2383842bb9e755d0f05cb9f0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:16.791144 21188 consensus_queue.cc:1035] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:16.791282 21151 consensus_queue.cc:1035] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:16.791896 21153 tablet_replica.cc:439] T cb3ee737d92549409d1221ce98ba7c41 P da34cf54bf864b62a671126b0b163fe6: TxnStatusTablet state changed. Reason: New leader f7b176d2383842bb9e755d0f05cb9f0c. Latest consensus state: current_term: 1 leader_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } } }
I20260924 23:11:16.791996 21153 tablet_replica.cc:442] T cb3ee737d92549409d1221ce98ba7c41 P da34cf54bf864b62a671126b0b163fe6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:11:16.792485 21155 tablet_replica.cc:439] T cb3ee737d92549409d1221ce98ba7c41 P 6bb78406254f4487b617881308a89902: TxnStatusTablet state changed. Reason: New leader f7b176d2383842bb9e755d0f05cb9f0c. Latest consensus state: current_term: 1 leader_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } } }
I20260924 23:11:16.792594 21155 tablet_replica.cc:442] T cb3ee737d92549409d1221ce98ba7c41 P 6bb78406254f4487b617881308a89902: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:11:16.792979 21151 tablet_replica.cc:439] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } } }
I20260924 23:11:16.793073 21151 tablet_replica.cc:442] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c: This TxnStatusTablet replica's current role is: LEADER
I20260924 23:11:16.793160 21151 tablet_replica.cc:439] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } } }
I20260924 23:11:16.793198 21151 tablet_replica.cc:442] T cb3ee737d92549409d1221ce98ba7c41 P f7b176d2383842bb9e755d0f05cb9f0c: This TxnStatusTablet replica's current role is: LEADER
I20260924 23:11:16.793610 21153 tablet_replica.cc:439] T cb3ee737d92549409d1221ce98ba7c41 P da34cf54bf864b62a671126b0b163fe6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } } }
I20260924 23:11:16.793695 21153 tablet_replica.cc:442] T cb3ee737d92549409d1221ce98ba7c41 P da34cf54bf864b62a671126b0b163fe6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:11:16.794333 21158 mvcc.cc:205] Tried to move back new op lower bound from 7333033888932556800 to 7333033888840257536. Current Snapshot: MvccSnapshot[applied={T|T < 7333033888932556800}]
I20260924 23:11:16.794646 21155 tablet_replica.cc:439] T cb3ee737d92549409d1221ce98ba7c41 P 6bb78406254f4487b617881308a89902: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } } }
I20260924 23:11:16.794770 21155 tablet_replica.cc:442] T cb3ee737d92549409d1221ce98ba7c41 P 6bb78406254f4487b617881308a89902: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 23:11:16.797266 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.794622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59790 (local address 127.12.212.1:38081)
0924 23:11:16.794664 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.794666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.794674 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:11:16.794725 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.794726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.794748 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.794822 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:11:16.795004 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.795542 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.796581 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.796763 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.796811 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.796957 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:11:16.797063 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:11:16.797122 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.797186 (+ 64us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.797206 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 23:11:16.802629 21143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.799846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:46951 (local address 127.12.212.2:46595)
0924 23:11:16.799888 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.799890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.799948 (+ 58us) server_negotiation.cc:407] Connection header received
0924 23:11:16.800031 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.800032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.800040 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.800083 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:11:16.800225 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.800701 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.801877 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.802079 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.802145 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.802381 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:11:16.802418 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:11:16.802470 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.802542 (+ 72us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.802566 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 23:11:16.802726 21197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.799751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:46595 (local address 127.12.212.1:46951)
0924 23:11:16.799893 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:16.799904 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:16.799962 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:16.800089 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:16.800090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:16.800091 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 23:11:16.800167 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.800169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.800776 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.800777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.801751 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.801754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.802164 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.802164 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.802237 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.802261 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 23:11:16.802484 (+ 223us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 23:11:16.802486 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 23:11:16.802540 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 23:11:16.802591 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":67,"threads_started":1}
I20260924 23:11:16.805351 20931 raft_consensus.cc:1261] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c [term 1 FOLLOWER]: Refusing update from remote peer da34cf54bf864b62a671126b0b163fe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:16.805377 21071 raft_consensus.cc:1261] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902 [term 1 FOLLOWER]: Refusing update from remote peer da34cf54bf864b62a671126b0b163fe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:16.805605 21153 consensus_queue.cc:1035] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:16.805747 21175 consensus_queue.cc:1035] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:16.814023 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.811209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:58619 (local address 127.12.212.1:38081)
0924 23:11:16.811325 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.811327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.811382 (+ 55us) server_negotiation.cc:407] Connection header received
0924 23:11:16.811486 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.811487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.811498 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.811536 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:11:16.811728 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.812250 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.813362 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.813602 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.813651 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.813837 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:11:16.813863 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:11:16.813893 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.813947 (+ 54us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.813962 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260924 23:11:16.814085 21202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.811165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:38081 (local address 127.12.212.2:58619)
0924 23:11:16.811320 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:16.811331 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:16.811408 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:16.811555 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:16.811556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:16.811557 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 23:11:16.811635 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.811637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.812326 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.812327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.813281 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:16.813284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.813669 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.813670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:16.813740 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.813757 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 23:11:16.813896 (+ 139us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 23:11:16.813898 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 23:11:16.813944 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 23:11:16.813988 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260924 23:11:16.824306 20633 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:11:16.824414 20633 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:11:16.825045 20633 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc: Sending DeleteTablet for 3 replicas of tablet bb22686c962e43d597c08c2719f0194d
I20260924 23:11:16.825314 21051 tablet_service.cc:1514] Processing DeleteTablet for tablet bb22686c962e43d597c08c2719f0194d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:16 UTC) from {username='oryx', principal='oryx/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:60194
I20260924 23:11:16.825409 21168 tablet_replica.cc:330] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902: stopping tablet replica
I20260924 23:11:16.825460 21168 raft_consensus.cc:2229] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:11:16.825548 21168 raft_consensus.cc:2258] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:16.825810 20757 tablet_service.cc:1514] Processing DeleteTablet for tablet bb22686c962e43d597c08c2719f0194d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:16 UTC) from {username='oryx', principal='oryx/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:59788
I20260924 23:11:16.825941 20911 tablet_service.cc:1514] Processing DeleteTablet for tablet bb22686c962e43d597c08c2719f0194d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:16 UTC) from {username='oryx', principal='oryx/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:36878
I20260924 23:11:16.826007 21167 tablet_replica.cc:330] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6: stopping tablet replica
I20260924 23:11:16.826040 21169 tablet_replica.cc:330] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c: stopping tablet replica
I20260924 23:11:16.826066 21167 raft_consensus.cc:2229] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:11:16.826081 21169 raft_consensus.cc:2229] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:11:16.826128 21169 raft_consensus.cc:2258] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:16.826169 21167 raft_consensus.cc:2258] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:16.826442 21169 ts_tablet_manager.cc:1905] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:16.826448 21167 ts_tablet_manager.cc:1905] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:16.826887 21168 ts_tablet_manager.cc:1905] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:16.828101 21167 ts_tablet_manager.cc:1918] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 23:11:16.828152 21167 log.cc:1196] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/wal/wals/bb22686c962e43d597c08c2719f0194d
I20260924 23:11:16.828364 21167 ts_tablet_manager.cc:1939] T bb22686c962e43d597c08c2719f0194d P da34cf54bf864b62a671126b0b163fe6: Deleting consensus metadata
I20260924 23:11:16.828397 21168 ts_tablet_manager.cc:1918] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 23:11:16.828449 21168 log.cc:1196] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/bb22686c962e43d597c08c2719f0194d
I20260924 23:11:16.828663 21168 ts_tablet_manager.cc:1939] T bb22686c962e43d597c08c2719f0194d P 6bb78406254f4487b617881308a89902: Deleting consensus metadata
I20260924 23:11:16.828914 20621 catalog_manager.cc:4841] TS da34cf54bf864b62a671126b0b163fe6 (127.12.212.1:38081): tablet bb22686c962e43d597c08c2719f0194d (table test-table [id=7cca3dc1dc40475696d63859978f18d9]) successfully deleted
I20260924 23:11:16.830735 20619 catalog_manager.cc:4841] TS 6bb78406254f4487b617881308a89902 (127.12.212.3:46471): tablet bb22686c962e43d597c08c2719f0194d (table test-table [id=7cca3dc1dc40475696d63859978f18d9]) successfully deleted
I20260924 23:11:16.832734 21169 ts_tablet_manager.cc:1918] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 23:11:16.832791 21169 log.cc:1196] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/bb22686c962e43d597c08c2719f0194d
I20260924 23:11:16.832978 21169 ts_tablet_manager.cc:1939] T bb22686c962e43d597c08c2719f0194d P f7b176d2383842bb9e755d0f05cb9f0c: Deleting consensus metadata
I20260924 23:11:16.833959 20621 catalog_manager.cc:4841] TS f7b176d2383842bb9e755d0f05cb9f0c (127.12.212.2:46595): tablet bb22686c962e43d597c08c2719f0194d (table test-table [id=7cca3dc1dc40475696d63859978f18d9]) successfully deleted
Sep 24 23:11:16 dist-test-slave-m0z6 krb5kdc[20589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 23:11:16 dist-test-slave-m0z6 krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:16 dist-test-slave-m0z6 krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
W20260924 23:11:16.865238 20703 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:16.858072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57816 (local address 127.12.212.62:35281)
0924 23:11:16.858123 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.858125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.858296 (+ 171us) server_negotiation.cc:407] Connection header received
0924 23:11:16.858354 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.858356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.858369 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.858437 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:16.859260 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.860065 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.861217 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.861493 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.861575 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.865164 (+ 3589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57816: BlockingRecv error: recv got EOF from 127.0.0.1:57816 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 24 23:11:16 dist-test-slave-m0z6 krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM
I20260924 23:11:16.875345 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.866141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57820 (local address 127.12.212.62:35281)
0924 23:11:16.866186 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.866188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.866455 (+ 267us) server_negotiation.cc:407] Connection header received
0924 23:11:16.866514 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.866515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.866526 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.866586 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:16.867238 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.868016 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.869044 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.869385 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.869461 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.872282 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:16.872284 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:16.872285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:16.872293 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:16.874016 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:16.874437 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:16.874438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:16.874439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:16.874506 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:16.874759 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:16.874760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:16.874761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:16.874913 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:16.875016 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.875164 (+ 148us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.875247 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 23:11:16.877231 20634 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57820:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:11:16.877406 20634 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:11:16.879319 21051 tablet_service.cc:1467] Processing CreateTablet for tablet 25e7bb0a89ee4876bbd47c5d2063709d (DEFAULT_TABLE table=test-table [id=345c2faaff854709991663969923786c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:16.879396 20757 tablet_service.cc:1467] Processing CreateTablet for tablet 25e7bb0a89ee4876bbd47c5d2063709d (DEFAULT_TABLE table=test-table [id=345c2faaff854709991663969923786c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:16.879463 21051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25e7bb0a89ee4876bbd47c5d2063709d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:16.879525 20757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25e7bb0a89ee4876bbd47c5d2063709d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:16.880928 20911 tablet_service.cc:1467] Processing CreateTablet for tablet 25e7bb0a89ee4876bbd47c5d2063709d (DEFAULT_TABLE table=test-table [id=345c2faaff854709991663969923786c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:16.881048 20911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25e7bb0a89ee4876bbd47c5d2063709d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:16.882364 21148 tablet_bootstrap.cc:492] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6: Bootstrap starting.
I20260924 23:11:16.882767 21148 tablet_bootstrap.cc:654] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:16.883105 21146 tablet_bootstrap.cc:492] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902: Bootstrap starting.
I20260924 23:11:16.883422 21146 tablet_bootstrap.cc:654] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:16.883673 21145 tablet_bootstrap.cc:492] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c: Bootstrap starting.
I20260924 23:11:16.884004 21145 tablet_bootstrap.cc:654] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:16.884595 21145 tablet_bootstrap.cc:492] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c: No bootstrap required, opened a new log
I20260924 23:11:16.884646 21145 ts_tablet_manager.cc:1397] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 23:11:16.884791 21145 raft_consensus.cc:348] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.884848 21148 tablet_bootstrap.cc:492] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6: No bootstrap required, opened a new log
I20260924 23:11:16.884867 21145 raft_consensus.cc:374] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:16.884887 21145 raft_consensus.cc:729] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7b176d2383842bb9e755d0f05cb9f0c, State: Initialized, Role: FOLLOWER
I20260924 23:11:16.884893 21148 ts_tablet_manager.cc:1397] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260924 23:11:16.884928 21145 consensus_queue.cc:260] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.885026 21145 ts_tablet_manager.cc:1428] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 23:11:16.885040 21148 raft_consensus.cc:348] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.885105 21148 raft_consensus.cc:374] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:16.885123 21148 raft_consensus.cc:729] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da34cf54bf864b62a671126b0b163fe6, State: Initialized, Role: FOLLOWER
I20260924 23:11:16.885164 21148 consensus_queue.cc:260] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.885239 21148 ts_tablet_manager.cc:1428] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:16.887018 21146 tablet_bootstrap.cc:492] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902: No bootstrap required, opened a new log
I20260924 23:11:16.887086 21146 ts_tablet_manager.cc:1397] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260924 23:11:16.887212 21146 raft_consensus.cc:348] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.887286 21146 raft_consensus.cc:374] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:16.887303 21146 raft_consensus.cc:729] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb78406254f4487b617881308a89902, State: Initialized, Role: FOLLOWER
I20260924 23:11:16.887342 21146 consensus_queue.cc:260] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.887426 21146 ts_tablet_manager.cc:1428] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 23:11:16.912482 21155 raft_consensus.cc:482] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:11:16.912603 21155 raft_consensus.cc:504] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.912763 21155 leader_election.cc:288] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da34cf54bf864b62a671126b0b163fe6 (127.12.212.1:38081), f7b176d2383842bb9e755d0f05cb9f0c (127.12.212.2:46595)
I20260924 23:11:16.913005 20931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e7bb0a89ee4876bbd47c5d2063709d" candidate_uuid: "6bb78406254f4487b617881308a89902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" is_pre_election: true
I20260924 23:11:16.913048 20778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e7bb0a89ee4876bbd47c5d2063709d" candidate_uuid: "6bb78406254f4487b617881308a89902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da34cf54bf864b62a671126b0b163fe6" is_pre_election: true
I20260924 23:11:16.913106 20931 raft_consensus.cc:2454] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bb78406254f4487b617881308a89902 in term 0.
I20260924 23:11:16.913142 20778 raft_consensus.cc:2454] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bb78406254f4487b617881308a89902 in term 0.
I20260924 23:11:16.913249 21005 leader_election.cc:302] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bb78406254f4487b617881308a89902, f7b176d2383842bb9e755d0f05cb9f0c; no voters:
I20260924 23:11:16.913365 21155 raft_consensus.cc:2781] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:11:16.913398 21155 raft_consensus.cc:482] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:11:16.913415 21155 raft_consensus.cc:3037] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:16.914315 21155 raft_consensus.cc:504] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.914461 21155 leader_election.cc:288] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [CANDIDATE]: Term 1 election: Requested vote from peers da34cf54bf864b62a671126b0b163fe6 (127.12.212.1:38081), f7b176d2383842bb9e755d0f05cb9f0c (127.12.212.2:46595)
I20260924 23:11:16.914759 20931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e7bb0a89ee4876bbd47c5d2063709d" candidate_uuid: "6bb78406254f4487b617881308a89902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7b176d2383842bb9e755d0f05cb9f0c"
I20260924 23:11:16.914742 20778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e7bb0a89ee4876bbd47c5d2063709d" candidate_uuid: "6bb78406254f4487b617881308a89902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da34cf54bf864b62a671126b0b163fe6"
I20260924 23:11:16.914834 20931 raft_consensus.cc:3037] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:16.914844 20778 raft_consensus.cc:3037] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:16.915581 20931 raft_consensus.cc:2454] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bb78406254f4487b617881308a89902 in term 1.
I20260924 23:11:16.915716 20778 raft_consensus.cc:2454] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bb78406254f4487b617881308a89902 in term 1.
I20260924 23:11:16.915758 21005 leader_election.cc:302] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bb78406254f4487b617881308a89902, f7b176d2383842bb9e755d0f05cb9f0c; no voters:
I20260924 23:11:16.915928 21155 raft_consensus.cc:2781] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:16.915968 21155 raft_consensus.cc:686] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [term 1 LEADER]: Becoming Leader. State: Replica: 6bb78406254f4487b617881308a89902, State: Running, Role: LEADER
I20260924 23:11:16.916008 21155 consensus_queue.cc:237] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.916571 20634 catalog_manager.cc:5495] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bb78406254f4487b617881308a89902 (127.12.212.3). New cstate: current_term: 1 leader_uuid: "6bb78406254f4487b617881308a89902" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } health_report { overall_health: HEALTHY } } }
I20260924 23:11:16.929690 21144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:16.926934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60206 (local address 127.12.212.3:46471)
0924 23:11:16.927007 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:16.927009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:16.927019 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:11:16.927047 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:16.927048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:16.927058 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:16.927114 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:11:16.927270 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.927985 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.929079 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:16.929282 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:16.929340 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:16.929497 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:11:16.929530 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:11:16.929561 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:16.929611 (+ 50us) server_negotiation.cc:299] Negotiation successful
0924 23:11:16.929628 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 23:11:16.930410 20931 raft_consensus.cc:1261] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c [term 1 FOLLOWER]: Refusing update from remote peer 6bb78406254f4487b617881308a89902: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:16.930447 20778 raft_consensus.cc:1261] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6 [term 1 FOLLOWER]: Refusing update from remote peer 6bb78406254f4487b617881308a89902: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:16.930651 21152 consensus_queue.cc:1035] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:16.930778 21155 consensus_queue.cc:1035] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:16.933307 20633 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:11:16.933404 20633 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:11:16.933951 20633 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc: Sending DeleteTablet for 3 replicas of tablet 25e7bb0a89ee4876bbd47c5d2063709d
I20260924 23:11:16.934239 21051 tablet_service.cc:1514] Processing DeleteTablet for tablet 25e7bb0a89ee4876bbd47c5d2063709d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:16 UTC) from {username='oryx', principal='oryx/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:60194
I20260924 23:11:16.934239 20757 tablet_service.cc:1514] Processing DeleteTablet for tablet 25e7bb0a89ee4876bbd47c5d2063709d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:16 UTC) from {username='oryx', principal='oryx/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:59788
I20260924 23:11:16.934345 21168 tablet_replica.cc:330] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902: stopping tablet replica
I20260924 23:11:16.934403 21168 raft_consensus.cc:2229] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:11:16.934456 21168 raft_consensus.cc:2258] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:16.934418 20633 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57820:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 23:11:16.934520 21167 tablet_replica.cc:330] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6: stopping tablet replica
W20260924 23:11:16.934566 20633 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:11:16.934569 21167 raft_consensus.cc:2229] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:11:16.934607 21167 raft_consensus.cc:2258] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:16.934937 20911 tablet_service.cc:1514] Processing DeleteTablet for tablet 25e7bb0a89ee4876bbd47c5d2063709d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:16 UTC) from {username='oryx', principal='oryx/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:36878
I20260924 23:11:16.934980 21167 ts_tablet_manager.cc:1905] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:16.935034 21169 tablet_replica.cc:330] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c: stopping tablet replica
I20260924 23:11:16.935081 21169 raft_consensus.cc:2229] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:11:16.935127 21169 raft_consensus.cc:2258] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:16.936141 20757 tablet_service.cc:1467] Processing CreateTablet for tablet c770399de6d14c3db35510a4eb4608d7 (DEFAULT_TABLE table=test-table [id=775fa6df26b24c0a888e6f20f32dba61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:16.936245 20757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c770399de6d14c3db35510a4eb4608d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:16.936471 20911 tablet_service.cc:1467] Processing CreateTablet for tablet c770399de6d14c3db35510a4eb4608d7 (DEFAULT_TABLE table=test-table [id=775fa6df26b24c0a888e6f20f32dba61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:16.936537 21168 ts_tablet_manager.cc:1905] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:16.936614 20911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c770399de6d14c3db35510a4eb4608d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:16.937778 21168 ts_tablet_manager.cc:1918] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:11:16.937829 21168 log.cc:1196] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/25e7bb0a89ee4876bbd47c5d2063709d
I20260924 23:11:16.937984 21168 ts_tablet_manager.cc:1939] T 25e7bb0a89ee4876bbd47c5d2063709d P 6bb78406254f4487b617881308a89902: Deleting consensus metadata
I20260924 23:11:16.938086 21145 tablet_bootstrap.cc:492] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c: Bootstrap starting.
I20260924 23:11:16.938205 21051 tablet_service.cc:1467] Processing CreateTablet for tablet c770399de6d14c3db35510a4eb4608d7 (DEFAULT_TABLE table=test-table [id=775fa6df26b24c0a888e6f20f32dba61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:16.938246 20619 catalog_manager.cc:4841] TS 6bb78406254f4487b617881308a89902 (127.12.212.3:46471): tablet 25e7bb0a89ee4876bbd47c5d2063709d (table test-table [id=345c2faaff854709991663969923786c]) successfully deleted
I20260924 23:11:16.938303 21051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c770399de6d14c3db35510a4eb4608d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:16.938421 21145 tablet_bootstrap.cc:654] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:16.938782 21169 ts_tablet_manager.cc:1905] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:16.939260 21146 tablet_bootstrap.cc:492] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902: Bootstrap starting.
I20260924 23:11:16.939571 21146 tablet_bootstrap.cc:654] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:16.939982 21146 tablet_bootstrap.cc:492] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902: No bootstrap required, opened a new log
I20260924 23:11:16.940012 21146 ts_tablet_manager.cc:1397] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260924 23:11:16.940092 21169 ts_tablet_manager.cc:1918] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:11:16.940101 21146 raft_consensus.cc:348] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.940138 21169 log.cc:1196] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/25e7bb0a89ee4876bbd47c5d2063709d
I20260924 23:11:16.940151 21146 raft_consensus.cc:374] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:16.940164 21146 raft_consensus.cc:729] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb78406254f4487b617881308a89902, State: Initialized, Role: FOLLOWER
I20260924 23:11:16.940198 21146 consensus_queue.cc:260] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.940277 21146 ts_tablet_manager.cc:1428] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:16.940317 21169 ts_tablet_manager.cc:1939] T 25e7bb0a89ee4876bbd47c5d2063709d P f7b176d2383842bb9e755d0f05cb9f0c: Deleting consensus metadata
I20260924 23:11:16.940768 20621 catalog_manager.cc:4841] TS f7b176d2383842bb9e755d0f05cb9f0c (127.12.212.2:46595): tablet 25e7bb0a89ee4876bbd47c5d2063709d (table test-table [id=345c2faaff854709991663969923786c]) successfully deleted
I20260924 23:11:16.940934 21145 tablet_bootstrap.cc:492] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c: No bootstrap required, opened a new log
I20260924 23:11:16.940973 21145 ts_tablet_manager.cc:1397] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260924 23:11:16.941154 21145 raft_consensus.cc:348] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.941213 21145 raft_consensus.cc:374] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:16.941234 21145 raft_consensus.cc:729] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7b176d2383842bb9e755d0f05cb9f0c, State: Initialized, Role: FOLLOWER
I20260924 23:11:16.941279 21145 consensus_queue.cc:260] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.941358 21145 ts_tablet_manager.cc:1428] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:16.941841 21167 ts_tablet_manager.cc:1918] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:11:16.941900 21167 log.cc:1196] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-0/wal/wals/25e7bb0a89ee4876bbd47c5d2063709d
I20260924 23:11:16.942097 21167 ts_tablet_manager.cc:1939] T 25e7bb0a89ee4876bbd47c5d2063709d P da34cf54bf864b62a671126b0b163fe6: Deleting consensus metadata
I20260924 23:11:16.942376 20621 catalog_manager.cc:4841] TS da34cf54bf864b62a671126b0b163fe6 (127.12.212.1:38081): tablet 25e7bb0a89ee4876bbd47c5d2063709d (table test-table [id=345c2faaff854709991663969923786c]) successfully deleted
I20260924 23:11:16.943553 21148 tablet_bootstrap.cc:492] T c770399de6d14c3db35510a4eb4608d7 P da34cf54bf864b62a671126b0b163fe6: Bootstrap starting.
I20260924 23:11:16.943889 21148 tablet_bootstrap.cc:654] T c770399de6d14c3db35510a4eb4608d7 P da34cf54bf864b62a671126b0b163fe6: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:16.944415 21148 tablet_bootstrap.cc:492] T c770399de6d14c3db35510a4eb4608d7 P da34cf54bf864b62a671126b0b163fe6: No bootstrap required, opened a new log
I20260924 23:11:16.944459 21148 ts_tablet_manager.cc:1397] T c770399de6d14c3db35510a4eb4608d7 P da34cf54bf864b62a671126b0b163fe6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 23:11:16.944578 21148 raft_consensus.cc:348] T c770399de6d14c3db35510a4eb4608d7 P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.944638 21148 raft_consensus.cc:374] T c770399de6d14c3db35510a4eb4608d7 P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:16.944655 21148 raft_consensus.cc:729] T c770399de6d14c3db35510a4eb4608d7 P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da34cf54bf864b62a671126b0b163fe6, State: Initialized, Role: FOLLOWER
I20260924 23:11:16.944695 21148 consensus_queue.cc:260] T c770399de6d14c3db35510a4eb4608d7 P da34cf54bf864b62a671126b0b163fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:16.944783 21148 ts_tablet_manager.cc:1428] T c770399de6d14c3db35510a4eb4608d7 P da34cf54bf864b62a671126b0b163fe6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:17.011443 21198 raft_consensus.cc:482] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:11:17.011543 21198 raft_consensus.cc:504] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:17.011704 21198 leader_election.cc:288] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da34cf54bf864b62a671126b0b163fe6 (127.12.212.1:38081), 6bb78406254f4487b617881308a89902 (127.12.212.3:46471)
I20260924 23:11:17.011929 21071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c770399de6d14c3db35510a4eb4608d7" candidate_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bb78406254f4487b617881308a89902" is_pre_election: true
I20260924 23:11:17.011961 20778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c770399de6d14c3db35510a4eb4608d7" candidate_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da34cf54bf864b62a671126b0b163fe6" is_pre_election: true
I20260924 23:11:17.012048 20778 raft_consensus.cc:2454] T c770399de6d14c3db35510a4eb4608d7 P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7b176d2383842bb9e755d0f05cb9f0c in term 0.
I20260924 23:11:17.012046 21071 raft_consensus.cc:2454] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7b176d2383842bb9e755d0f05cb9f0c in term 0.
I20260924 23:11:17.012259 20863 leader_election.cc:302] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bb78406254f4487b617881308a89902, f7b176d2383842bb9e755d0f05cb9f0c; no voters:
I20260924 23:11:17.012384 21198 raft_consensus.cc:2781] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:11:17.012428 21198 raft_consensus.cc:482] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:11:17.012447 21198 raft_consensus.cc:3037] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:17.013212 21198 raft_consensus.cc:504] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:17.013417 21198 leader_election.cc:288] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [CANDIDATE]: Term 1 election: Requested vote from peers da34cf54bf864b62a671126b0b163fe6 (127.12.212.1:38081), 6bb78406254f4487b617881308a89902 (127.12.212.3:46471)
I20260924 23:11:17.013608 21071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c770399de6d14c3db35510a4eb4608d7" candidate_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bb78406254f4487b617881308a89902"
I20260924 23:11:17.013698 21071 raft_consensus.cc:3037] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:17.013675 20778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c770399de6d14c3db35510a4eb4608d7" candidate_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da34cf54bf864b62a671126b0b163fe6"
I20260924 23:11:17.013773 20778 raft_consensus.cc:3037] T c770399de6d14c3db35510a4eb4608d7 P da34cf54bf864b62a671126b0b163fe6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:17.014375 20778 raft_consensus.cc:2454] T c770399de6d14c3db35510a4eb4608d7 P da34cf54bf864b62a671126b0b163fe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7b176d2383842bb9e755d0f05cb9f0c in term 1.
I20260924 23:11:17.014400 21071 raft_consensus.cc:2454] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7b176d2383842bb9e755d0f05cb9f0c in term 1.
I20260924 23:11:17.014573 20863 leader_election.cc:302] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bb78406254f4487b617881308a89902, f7b176d2383842bb9e755d0f05cb9f0c; no voters:
I20260924 23:11:17.014716 21198 raft_consensus.cc:2781] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:17.014781 21198 raft_consensus.cc:686] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [term 1 LEADER]: Becoming Leader. State: Replica: f7b176d2383842bb9e755d0f05cb9f0c, State: Running, Role: LEADER
I20260924 23:11:17.014844 21198 consensus_queue.cc:237] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } }
I20260924 23:11:17.015455 20633 catalog_manager.cc:5495] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c reported cstate change: term changed from 0 to 1, leader changed from <none> to f7b176d2383842bb9e755d0f05cb9f0c (127.12.212.2). New cstate: current_term: 1 leader_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7b176d2383842bb9e755d0f05cb9f0c" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 } health_report { overall_health: UNKNOWN } } }
I20260924 23:11:17.021667 21143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:17.019563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36894 (local address 127.12.212.2:46595)
0924 23:11:17.019604 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:17.019606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:17.019629 (+ 23us) server_negotiation.cc:407] Connection header received
0924 23:11:17.019680 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:17.019680 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:17.019689 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:17.019736 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:11:17.019819 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:17.020288 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:17.021050 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:17.021232 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:17.021285 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:17.021466 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:11:17.021494 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:11:17.021532 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:17.021586 (+ 54us) server_negotiation.cc:299] Negotiation successful
0924 23:11:17.021605 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 23:11:17.025964 21143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:17.023960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:35129 (local address 127.12.212.2:46595)
0924 23:11:17.024005 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:17.024007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:17.024014 (+ 7us) server_negotiation.cc:407] Connection header received
0924 23:11:17.024062 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:17.024063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:17.024071 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:17.024111 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:11:17.024207 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:17.024713 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:17.025380 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:17.025560 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:17.025611 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:17.025743 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:11:17.025773 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:11:17.025811 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:17.025861 (+ 50us) server_negotiation.cc:299] Negotiation successful
0924 23:11:17.025878 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":28}
I20260924 23:11:17.025972 21202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:17.023868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:46595 (local address 127.12.212.2:35129)
0924 23:11:17.023920 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:17.023970 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:17.024011 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:17.024116 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:17.024117 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:17.024118 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 23:11:17.024155 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:17.024156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:17.024775 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:17.024776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:17.025314 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:17.025315 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:17.025616 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:17.025617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:17.025684 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:17.025689 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 23:11:17.025821 (+ 132us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 23:11:17.025822 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 23:11:17.025861 (+ 39us) client_negotiation.cc:240] Negotiation successful
0924 23:11:17.025906 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 23:11:17.028489 21071 raft_consensus.cc:1261] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902 [term 1 FOLLOWER]: Refusing update from remote peer f7b176d2383842bb9e755d0f05cb9f0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:17.028717 21188 consensus_queue.cc:1035] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bb78406254f4487b617881308a89902" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:17.029073 20778 raft_consensus.cc:1261] T c770399de6d14c3db35510a4eb4608d7 P da34cf54bf864b62a671126b0b163fe6 [term 1 FOLLOWER]: Refusing update from remote peer f7b176d2383842bb9e755d0f05cb9f0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:17.029273 21188 consensus_queue.cc:1035] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "da34cf54bf864b62a671126b0b163fe6" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 38081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:17.047515 20634 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:11:17.047652 20634 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:11:17.048339 20634 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0014010946794434a6979a1a749145cc: Sending DeleteTablet for 3 replicas of tablet c770399de6d14c3db35510a4eb4608d7
I20260924 23:11:17.048667 20757 tablet_service.cc:1514] Processing DeleteTablet for tablet c770399de6d14c3db35510a4eb4608d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:17 UTC) from {username='oryx', principal='oryx/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:59788
I20260924 23:11:17.048666 21051 tablet_service.cc:1514] Processing DeleteTablet for tablet c770399de6d14c3db35510a4eb4608d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:17 UTC) from {username='oryx', principal='oryx/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:60194
I20260924 23:11:17.048768 21167 tablet_replica.cc:330] T c770399de6d14c3db35510a4eb4608d7 P da34cf54bf864b62a671126b0b163fe6: stopping tablet replica
I20260924 23:11:17.048768 21168 tablet_replica.cc:330] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902: stopping tablet replica
I20260924 23:11:17.048816 21167 raft_consensus.cc:2229] T c770399de6d14c3db35510a4eb4608d7 P da34cf54bf864b62a671126b0b163fe6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:11:17.048827 21168 raft_consensus.cc:2229] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:11:17.048852 21167 raft_consensus.cc:2258] T c770399de6d14c3db35510a4eb4608d7 P da34cf54bf864b62a671126b0b163fe6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:17.048862 21168 raft_consensus.cc:2258] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:17.048966 20911 tablet_service.cc:1514] Processing DeleteTablet for tablet c770399de6d14c3db35510a4eb4608d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:17 UTC) from {username='oryx', principal='oryx/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:36878
I20260924 23:11:17.049042 21169 tablet_replica.cc:330] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c: stopping tablet replica
I20260924 23:11:17.049089 21169 raft_consensus.cc:2229] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:11:17.049187 21169 raft_consensus.cc:2258] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:17.049467 21168 ts_tablet_manager.cc:1905] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:17.049530 21169 ts_tablet_manager.cc:1905] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:17.051260 21169 ts_tablet_manager.cc:1918] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 23:11:17.051333 21169 log.cc:1196] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/c770399de6d14c3db35510a4eb4608d7
I20260924 23:11:17.051334 21168 ts_tablet_manager.cc:1918] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 23:11:17.051394 21168 log.cc:1196] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/c770399de6d14c3db35510a4eb4608d7
I20260924 23:11:17.051534 21169 ts_tablet_manager.cc:1939] T c770399de6d14c3db35510a4eb4608d7 P f7b176d2383842bb9e755d0f05cb9f0c: Deleting consensus metadata
I20260924 23:11:17.051736 21168 ts_tablet_manager.cc:1939] T c770399de6d14c3db35510a4eb4608d7 P 6bb78406254f4487b617881308a89902: Deleting consensus metadata
I20260924 23:11:17.052105 20619 catalog_manager.cc:4841] TS 6bb78406254f4487b617881308a89902 (127.12.212.3:46471): tablet c770399de6d14c3db35510a4eb4608d7 (table test-table [id=775fa6df26b24c0a888e6f20f32dba61]) successfully deleted
I20260924 23:11:17.052140 20621 catalog_manager.cc:4841] TS f7b176d2383842bb9e755d0f05cb9f0c (127.12.212.2:46595): tablet c770399de6d14c3db35510a4eb4608d7 (table test-table [id=775fa6df26b24c0a888e6f20f32dba61]) successfully deleted
I20260924 23:11:17.052451 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 20706
W20260924 23:11:17.058138 20621 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.212.1:38081 (error 108)
I20260924 23:11:17.058487 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 20846
I20260924 23:11:17.060182 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:17.058539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:38081 (local address 127.0.0.1:59800)
0924 23:11:17.060078 (+ 1539us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:17.060129 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.1:38081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1518,"thread_start_us":52,"threads_started":1}
W20260924 23:11:17.060431 20621 catalog_manager.cc:4584] TS da34cf54bf864b62a671126b0b163fe6 (127.12.212.1:38081): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c770399de6d14c3db35510a4eb4608d7: Network error: Client connection negotiation failed: client connection to 127.12.212.1:38081: connect: Connection refused (error 111)
I20260924 23:11:17.064052 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 20986
I20260924 23:11:17.071092 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 20602
2026-09-24T23:11:17Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2455 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:11:17 dist-test-slave-m0z6 krb5kdc[21237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:11:17 dist-test-slave-m0z6 krb5kdc[21237](info): set up 2 sockets
Sep 24 23:11:17 dist-test-slave-m0z6 krb5kdc[21237](info): commencing operation
krb5kdc: starting...
W20260924 23:11:18.036139 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.945s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:11:18 dist-test-slave-m0z6 krb5kdc[21237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:11:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:11:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:11:18.154002 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:46853
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:33493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:46853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:18.235977 21250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:18.236235 21250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:18.236260 21250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:18.237614 21250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:11:18.237677 21250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:18.237690 21250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:18.237702 21250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:11:18.237713 21250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:11:18.239436 21250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:33493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:46853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:46853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:18.239723 21250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:18.239972 21250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:18.245546 21256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:18.245564 21258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:18.245564 21255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:18.245671 21250 server_base.cc:1034] running on GCE node
I20260924 23:11:18.246141 21250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:18.246469 21250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:18.247655 21250 hybrid_clock.cc:648] HybridClock initialized: now 1790291478247628 us; error 40 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-24T23:11:18Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1181 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:11:18 dist-test-slave-m0z6 krb5kdc[21263](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:11:18 dist-test-slave-m0z6 krb5kdc[21263](info): set up 2 sockets
Sep 24 23:11:18 dist-test-slave-m0z6 krb5kdc[21263](info): commencing operation
krb5kdc: starting...
W20260924 23:11:19.219074 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.947s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:11:19 dist-test-slave-m0z6 krb5kdc[21263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:11:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:11:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:11:19.337815 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:37739
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:34741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:37739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:19.418623 21276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:19.418895 21276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:19.418943 21276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:19.420183 21276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:11:19.420230 21276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:19.420243 21276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:19.420255 21276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:11:19.420266 21276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:11:19.421890 21276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:34741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:37739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:37739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:19.422147 21276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:19.422374 21276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:11:19.427264 21276 server_base.cc:1034] running on GCE node
W20260924 23:11:19.427234 21284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:19.427224 21281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:19.427234 21282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:19.427688 21276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:19.428009 21276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:19.429178 21276 hybrid_clock.cc:648] HybridClock initialized: now 1790291479429163 us; error 36 us; skew 500 ppm
Sep 24 23:11:19 dist-test-slave-m0z6 krb5kdc[21263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291479, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:19.431953 21276 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:11:19.432435 21276 webserver.cc:460] Webserver started at http://127.12.212.62:38167/ using document root <none> and password file <none>
I20260924 23:11:19.432663 21276 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:19.432716 21276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:19.432855 21276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:19.433957 21276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "c5a015c65488448c93ea38671983817c"
format_stamp: "Formatted at 2026-09-24 23:11:19 on dist-test-slave-m0z6"
server_key: "a71e37916918b9ba4b15324e73ee703e"
server_key_iv: "1e17bb39fdf3c6c43a76c5d7b83fecc9"
server_key_version: "encryptionkey@0"
I20260924 23:11:19.434306 21276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "c5a015c65488448c93ea38671983817c"
format_stamp: "Formatted at 2026-09-24 23:11:19 on dist-test-slave-m0z6"
server_key: "a71e37916918b9ba4b15324e73ee703e"
server_key_iv: "1e17bb39fdf3c6c43a76c5d7b83fecc9"
server_key_version: "encryptionkey@0"
I20260924 23:11:19.435792 21276 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:11:19.436661 21291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:19.436820 21276 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:11:19.436964 21276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "c5a015c65488448c93ea38671983817c"
format_stamp: "Formatted at 2026-09-24 23:11:19 on dist-test-slave-m0z6"
server_key: "a71e37916918b9ba4b15324e73ee703e"
server_key_iv: "1e17bb39fdf3c6c43a76c5d7b83fecc9"
server_key_version: "encryptionkey@0"
I20260924 23:11:19.437031 21276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:19.444195 21276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:19.446309 21276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:19.446449 21276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:19.450214 21276 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:37739
I20260924 23:11:19.450263 21343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:37739 every 8 connection(s)
I20260924 23:11:19.450589 21276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:11:19.451501 21344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:19.452492 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 21276
I20260924 23:11:19.452610 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:11:19.452796 13136 external_mini_cluster.cc:1428] Setting key 8d341dbb43329390613f186459c45a14
I20260924 23:11:19.454171 21344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c: Bootstrap starting.
I20260924 23:11:19.454943 21344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:19.455209 21344 log.cc:824] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:19.455873 21344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c: No bootstrap required, opened a new log
I20260924 23:11:19.457613 21344 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a015c65488448c93ea38671983817c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37739 } }
I20260924 23:11:19.457746 21344 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:19.457767 21344 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5a015c65488448c93ea38671983817c, State: Initialized, Role: FOLLOWER
I20260924 23:11:19.457875 21344 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a015c65488448c93ea38671983817c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37739 } }
I20260924 23:11:19.457948 21344 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:11:19.457971 21344 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:11:19.458014 21344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:19.458508 21344 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a015c65488448c93ea38671983817c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37739 } }
I20260924 23:11:19.458600 21344 leader_election.cc:302] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5a015c65488448c93ea38671983817c; no voters:
I20260924 23:11:19.458791 21344 leader_election.cc:288] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:11:19.458824 21349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:19.458930 21349 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [term 1 LEADER]: Becoming Leader. State: Replica: c5a015c65488448c93ea38671983817c, State: Running, Role: LEADER
I20260924 23:11:19.459012 21349 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a015c65488448c93ea38671983817c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37739 } }
I20260924 23:11:19.459066 21344 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [sys.catalog]: configured and running, proceeding with master startup.
Sep 24 23:11:19 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:19.459318 21349 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5a015c65488448c93ea38671983817c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a015c65488448c93ea38671983817c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37739 } } }
I20260924 23:11:19.459481 21349 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:19.459656 21349 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5a015c65488448c93ea38671983817c. Latest consensus state: current_term: 1 leader_uuid: "c5a015c65488448c93ea38671983817c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a015c65488448c93ea38671983817c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37739 } } }
I20260924 23:11:19.459721 21349 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:19.459976 21357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:11:19.460453 21357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:11:19.462073 21357 catalog_manager.cc:1347] Generated new cluster ID: 9493ed3a6b8740cdaa0bd7d9c77e29d6
I20260924 23:11:19.462126 21357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:11:19.463383 21347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:19.453323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56904 (local address 127.12.212.62:37739)
0924 23:11:19.453479 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:19.453489 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:19.453516 (+ 27us) server_negotiation.cc:407] Connection header received
0924 23:11:19.454000 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:19.454007 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:19.454056 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:19.454173 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:19.454947 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:19.456009 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:19.456728 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:19.457135 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:19.457231 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:19.459685 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:19.459689 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:19.459689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:19.459701 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:19.461823 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:19.462195 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:19.462199 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:19.462199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:19.462305 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:19.462483 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:19.462484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:19.462484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:19.462776 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:19.462911 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:19.463028 (+ 117us) server_negotiation.cc:299] Negotiation successful
0924 23:11:19.463163 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
I20260924 23:11:19.490558 21357 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:11:19.491189 21357 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:11:19.500850 21357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c: Generated new TSK 0
I20260924 23:11:19.501114 21357 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
I20260924 23:11:19.534085 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:37739
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:34741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:19.615525 21370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:19.615721 21370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:19.615743 21370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:19.616866 21370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:19.616900 21370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:19.616925 21370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:11:19.618273 21370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:34741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:37739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:19.618462 21370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:19.618631 21370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:19.623361 21378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:19.623389 21375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:19.623361 21376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:19.623406 21370 server_base.cc:1034] running on GCE node
I20260924 23:11:19.623828 21370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:19.624075 21370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:19.625223 21370 hybrid_clock.cc:648] HybridClock initialized: now 1790291479625200 us; error 37 us; skew 500 ppm
Sep 24 23:11:19 dist-test-slave-m0z6 krb5kdc[21263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291479, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:19.631480 21370 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:11:19.631925 21370 webserver.cc:460] Webserver started at http://127.12.212.1:42375/ using document root <none> and password file <none>
I20260924 23:11:19.632138 21370 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:19.632181 21370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:19.632305 21370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:19.633327 21370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "d8ab630f4440400da09554be6062e132"
format_stamp: "Formatted at 2026-09-24 23:11:19 on dist-test-slave-m0z6"
server_key: "c04bd4af8d85f15aa1b5de0259aea950"
server_key_iv: "8d65b3f94b493cd238c55d32eb4a0f80"
server_key_version: "encryptionkey@0"
I20260924 23:11:19.633634 21370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "d8ab630f4440400da09554be6062e132"
format_stamp: "Formatted at 2026-09-24 23:11:19 on dist-test-slave-m0z6"
server_key: "c04bd4af8d85f15aa1b5de0259aea950"
server_key_iv: "8d65b3f94b493cd238c55d32eb4a0f80"
server_key_version: "encryptionkey@0"
I20260924 23:11:19.634969 21370 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 23:11:19.635808 21385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:19.636030 21370 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:19.636107 21370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "d8ab630f4440400da09554be6062e132"
format_stamp: "Formatted at 2026-09-24 23:11:19 on dist-test-slave-m0z6"
server_key: "c04bd4af8d85f15aa1b5de0259aea950"
server_key_iv: "8d65b3f94b493cd238c55d32eb4a0f80"
server_key_version: "encryptionkey@0"
I20260924 23:11:19.636183 21370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:19.662002 21370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:19.664161 21370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:19.664331 21370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:19.664566 21370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:19.664861 21370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:19.664893 21370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:19.664923 21370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:19.664947 21370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:19.670466 21370 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:34271
I20260924 23:11:19.670524 21498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:34271 every 8 connection(s)
I20260924 23:11:19.670854 21370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
Sep 24 23:11:19 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291479, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:19.679253 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 21370
I20260924 23:11:19.679355 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:11:19.679553 13136 external_mini_cluster.cc:1428] Setting key ea61fe85a7afdb708b9ff4287384837a
I20260924 23:11:19.679819 21347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:19.671528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:33747 (local address 127.12.212.62:37739)
0924 23:11:19.671581 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:19.671583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:19.671768 (+ 185us) server_negotiation.cc:407] Connection header received
0924 23:11:19.672258 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:19.672259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:19.672268 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:19.672334 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:19.673222 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:19.673728 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:19.674756 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:19.674983 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:19.675041 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:19.677089 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:19.677090 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:19.677091 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:19.677097 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:19.678331 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:19.678799 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:19.678800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:19.678800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:19.678867 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:19.679087 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:19.679088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:19.679088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:19.679217 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:19.679277 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:19.679633 (+ 356us) server_negotiation.cc:299] Negotiation successful
0924 23:11:19.679730 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 23:11:19.680015 21501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:19.671504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.1:33747)
0924 23:11:19.671651 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:19.671693 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:19.672179 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:19.672392 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:19.672395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:19.672942 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:19.673136 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:19.673141 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:19.673818 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:19.673819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:19.674653 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:19.674655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:19.675084 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:19.675085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:19.675200 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:19.675648 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:19.675656 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:19.676996 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:19.678413 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:19.678419 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:19.678421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:19.678623 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:19.678925 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:19.678926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:19.678926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:19.679027 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:19.679293 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:19.679300 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:19.679502 (+ 202us) client_negotiation.cc:769] Sending connection context
0924 23:11:19.679602 (+ 100us) client_negotiation.cc:240] Negotiation successful
0924 23:11:19.679763 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
I20260924 23:11:19.680526 21499 heartbeater.cc:347] Connected to a master server at 127.12.212.62:37739
I20260924 23:11:19.680620 21499 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:19.680795 21499 heartbeater.cc:511] Master 127.12.212.62:37739 requested a full tablet report, sending...
I20260924 23:11:19.681365 21308 ts_manager.cc:194] Registered new tserver with Master: d8ab630f4440400da09554be6062e132 (127.12.212.1:34271)
I20260924 23:11:19.682034 21308 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:33747
WARNING: no policy specified for kudu/127.12.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
I20260924 23:11:19.702613 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:37739
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:34741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:19.781710 21504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:19.781934 21504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:19.781957 21504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:19.783154 21504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:19.783201 21504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:19.783231 21504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:11:19.784677 21504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:34741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:37739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:19.784891 21504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:19.785082 21504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:19.790016 21512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:19.790021 21510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:19.790016 21509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:19.790233 21504 server_base.cc:1034] running on GCE node
I20260924 23:11:19.790467 21504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:19.790728 21504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:19.791905 21504 hybrid_clock.cc:648] HybridClock initialized: now 1790291479791890 us; error 32 us; skew 500 ppm
Sep 24 23:11:19 dist-test-slave-m0z6 krb5kdc[21263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291479, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:19.794402 21504 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:11:19.794854 21504 webserver.cc:460] Webserver started at http://127.12.212.2:45381/ using document root <none> and password file <none>
I20260924 23:11:19.795050 21504 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:19.795097 21504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:19.795214 21504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:19.796245 21504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "89b79c99b28e4dbf876574f8ea6f9e56"
format_stamp: "Formatted at 2026-09-24 23:11:19 on dist-test-slave-m0z6"
server_key: "451fd5f2fc5093a6adcfa94fdc4afa89"
server_key_iv: "4dcd97db3504bf4dc49bfdafeebe95b4"
server_key_version: "encryptionkey@0"
I20260924 23:11:19.796572 21504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "89b79c99b28e4dbf876574f8ea6f9e56"
format_stamp: "Formatted at 2026-09-24 23:11:19 on dist-test-slave-m0z6"
server_key: "451fd5f2fc5093a6adcfa94fdc4afa89"
server_key_iv: "4dcd97db3504bf4dc49bfdafeebe95b4"
server_key_version: "encryptionkey@0"
I20260924 23:11:19.797794 21504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:11:19.798564 21519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:19.798750 21504 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:19.798842 21504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "89b79c99b28e4dbf876574f8ea6f9e56"
format_stamp: "Formatted at 2026-09-24 23:11:19 on dist-test-slave-m0z6"
server_key: "451fd5f2fc5093a6adcfa94fdc4afa89"
server_key_iv: "4dcd97db3504bf4dc49bfdafeebe95b4"
server_key_version: "encryptionkey@0"
I20260924 23:11:19.798915 21504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:19.810389 21504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:19.812639 21504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:19.812786 21504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:19.813004 21504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:19.813311 21504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:19.813345 21504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:19.813375 21504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:19.813391 21504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:19.819209 21504 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:41513
I20260924 23:11:19.819269 21632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:41513 every 8 connection(s)
I20260924 23:11:19.819592 21504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
Sep 24 23:11:19 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291479, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:19.827328 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 21504
I20260924 23:11:19.827441 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:11:19.827620 13136 external_mini_cluster.cc:1428] Setting key 6f35ffd8d67ab98c87e58365f660d0a3
I20260924 23:11:19.829144 21347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:19.820436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:47031 (local address 127.12.212.62:37739)
0924 23:11:19.820526 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:19.820528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:19.820701 (+ 173us) server_negotiation.cc:407] Connection header received
0924 23:11:19.821243 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:19.821245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:19.821256 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:19.821316 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:19.822291 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:19.822905 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:19.823919 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:19.824113 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:19.824174 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:19.826511 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:19.826512 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:19.826513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:19.826521 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:19.827866 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:19.828243 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:19.828244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:19.828245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:19.828298 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:19.828522 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:19.828523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:19.828523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:19.828630 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:19.828684 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:19.828965 (+ 281us) server_negotiation.cc:299] Negotiation successful
0924 23:11:19.829048 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260924 23:11:19.829316 21635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:19.820375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.2:47031)
0924 23:11:19.820576 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:19.820617 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:19.821128 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:19.821392 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:19.821395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:19.821981 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:19.822186 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:19.822191 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:19.822990 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:19.822991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:19.823817 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:19.823820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:19.824215 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:19.824217 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:19.824367 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:19.825001 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:19.825010 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:19.826414 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:19.827949 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:19.827957 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:19.827960 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:19.828179 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:19.828358 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:19.828359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:19.828359 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:19.828467 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:19.828683 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:19.828690 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:19.828829 (+ 139us) client_negotiation.cc:769] Sending connection context
0924 23:11:19.828934 (+ 105us) client_negotiation.cc:240] Negotiation successful
0924 23:11:19.829095 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":81,"threads_started":1}
I20260924 23:11:19.829723 21633 heartbeater.cc:347] Connected to a master server at 127.12.212.62:37739
I20260924 23:11:19.829818 21633 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:19.829988 21633 heartbeater.cc:511] Master 127.12.212.62:37739 requested a full tablet report, sending...
I20260924 23:11:19.830358 21308 ts_manager.cc:194] Registered new tserver with Master: 89b79c99b28e4dbf876574f8ea6f9e56 (127.12.212.2:41513)
I20260924 23:11:19.830860 21308 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:47031
WARNING: no policy specified for kudu/127.12.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
I20260924 23:11:19.853621 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:37739
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:34741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:19.932538 21638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:19.932762 21638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:19.932787 21638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:19.933966 21638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:19.934008 21638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:19.934034 21638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:11:19.935563 21638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:34741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:37739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:19.935773 21638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:19.935961 21638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:19.940821 21646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:19.940831 21643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:19.940821 21644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:19.941037 21638 server_base.cc:1034] running on GCE node
I20260924 23:11:19.941262 21638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:19.941469 21638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:19.942647 21638 hybrid_clock.cc:648] HybridClock initialized: now 1790291479942634 us; error 27 us; skew 500 ppm
Sep 24 23:11:19 dist-test-slave-m0z6 krb5kdc[21263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291479, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:19.945278 21638 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:11:19.945720 21638 webserver.cc:460] Webserver started at http://127.12.212.3:44199/ using document root <none> and password file <none>
I20260924 23:11:19.945966 21638 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:19.946017 21638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:19.946143 21638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:19.947222 21638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "3c2b6d479c144b0caed21854f96d904e"
format_stamp: "Formatted at 2026-09-24 23:11:19 on dist-test-slave-m0z6"
server_key: "9b79ce323639acea2e35ce3ccc509ca0"
server_key_iv: "d9ab4ec8ebdaaaa50763dbf6ed8de13b"
server_key_version: "encryptionkey@0"
I20260924 23:11:19.947577 21638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "3c2b6d479c144b0caed21854f96d904e"
format_stamp: "Formatted at 2026-09-24 23:11:19 on dist-test-slave-m0z6"
server_key: "9b79ce323639acea2e35ce3ccc509ca0"
server_key_iv: "d9ab4ec8ebdaaaa50763dbf6ed8de13b"
server_key_version: "encryptionkey@0"
I20260924 23:11:19.948848 21638 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:11:19.949560 21653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:19.949716 21638 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:19.949846 21638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "3c2b6d479c144b0caed21854f96d904e"
format_stamp: "Formatted at 2026-09-24 23:11:19 on dist-test-slave-m0z6"
server_key: "9b79ce323639acea2e35ce3ccc509ca0"
server_key_iv: "d9ab4ec8ebdaaaa50763dbf6ed8de13b"
server_key_version: "encryptionkey@0"
I20260924 23:11:19.949910 21638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:19.957441 21638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:19.959551 21638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:19.959684 21638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:19.959903 21638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:19.960295 21638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:19.960355 21638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:19.960384 21638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:19.960408 21638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:19.965961 21638 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:44665
I20260924 23:11:19.966032 21766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:44665 every 8 connection(s)
I20260924 23:11:19.966336 21638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:11:19.968315 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 21638
I20260924 23:11:19.968425 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:11:19.968585 13136 external_mini_cluster.cc:1428] Setting key b153e4181c1386c0041fe416e67ab68a
Sep 24 23:11:19 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291479, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:19.976351 21347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:19.967117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:54913 (local address 127.12.212.62:37739)
0924 23:11:19.967199 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:19.967201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:19.967356 (+ 155us) server_negotiation.cc:407] Connection header received
0924 23:11:19.967895 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:19.967897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:19.967909 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:19.967968 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:19.969124 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:19.969794 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:19.970878 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:19.971065 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:19.971118 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:19.973441 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:19.973442 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:19.973442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:19.973450 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:19.974885 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:19.975285 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:19.975286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:19.975287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:19.975349 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:19.975664 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:19.975665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:19.975666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:19.975796 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:19.975903 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:19.976207 (+ 304us) server_negotiation.cc:299] Negotiation successful
0924 23:11:19.976278 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 23:11:19.976634 21769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:19.967046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.3:54913)
0924 23:11:19.967205 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:19.967252 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:19.967785 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:19.968039 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:19.968042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:19.968833 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:19.969041 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:19.969047 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:19.969878 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:19.969879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:19.970744 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:19.970748 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:19.971160 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:19.971161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:19.971285 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:19.971845 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:19.971854 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:19.973344 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:19.974966 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:19.974974 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:19.974976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:19.975208 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:19.975446 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:19.975447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:19.975447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:19.975579 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:19.975918 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:19.975925 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:19.976082 (+ 157us) client_negotiation.cc:769] Sending connection context
0924 23:11:19.976182 (+ 100us) client_negotiation.cc:240] Negotiation successful
0924 23:11:19.976320 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260924 23:11:19.977273 21767 heartbeater.cc:347] Connected to a master server at 127.12.212.62:37739
I20260924 23:11:19.977382 21767 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:19.977581 21767 heartbeater.cc:511] Master 127.12.212.62:37739 requested a full tablet report, sending...
I20260924 23:11:19.978017 21308 ts_manager.cc:194] Registered new tserver with Master: 3c2b6d479c144b0caed21854f96d904e (127.12.212.3:44665)
I20260924 23:11:19.978386 21308 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:54913
I20260924 23:11:19.980546 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.1.keytab.
Entry for principal oryx/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.1.keytab.
WARNING: no policy specified for oryx/127.12.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.2.keytab.
Entry for principal oryx/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.2.keytab.
WARNING: no policy specified for oryx/127.12.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.3.keytab.
Entry for principal oryx/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.3.keytab.
I20260924 23:11:20.062891 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 21370
I20260924 23:11:20.066671 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 21504
I20260924 23:11:20.070554 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 21638
I20260924 23:11:20.074230 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 21276
I20260924 23:11:20.078970 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:37739
--webserver_interface=127.12.212.62
--webserver_port=38167
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:34741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:37739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:20.159960 21776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:20.160188 21776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:20.160219 21776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:20.161491 21776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:11:20.161549 21776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:20.161568 21776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:20.161584 21776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:11:20.161599 21776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:11:20.163271 21776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:34741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:37739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:37739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=38167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:20.163545 21776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:20.163756 21776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:20.168614 21784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:20.168629 21782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:20.168622 21781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:20.168771 21776 server_base.cc:1034] running on GCE node
I20260924 23:11:20.169109 21776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:20.169414 21776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:20.170574 21776 hybrid_clock.cc:648] HybridClock initialized: now 1790291480170561 us; error 32 us; skew 500 ppm
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291480, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:20.173420 21776 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:11:20.173898 21776 webserver.cc:460] Webserver started at http://127.12.212.62:38167/ using document root <none> and password file <none>
I20260924 23:11:20.174136 21776 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:20.174244 21776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:20.175848 21776 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:20.176682 21791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:20.176960 21776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:20.177068 21776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "c5a015c65488448c93ea38671983817c"
format_stamp: "Formatted at 2026-09-24 23:11:19 on dist-test-slave-m0z6"
server_key: "a71e37916918b9ba4b15324e73ee703e"
server_key_iv: "1e17bb39fdf3c6c43a76c5d7b83fecc9"
server_key_version: "encryptionkey@0"
I20260924 23:11:20.177444 21776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:20.186975 21776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:20.189108 21776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:20.189225 21776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:20.193143 21776 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:37739
I20260924 23:11:20.193177 21843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:37739 every 8 connection(s)
I20260924 23:11:20.193547 21776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:11:20.195106 21844 sys_catalog.cc:263] Verifying existing consensus state
I20260924 23:11:20.195739 21844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c: Bootstrap starting.
I20260924 23:11:20.203639 21844 log.cc:824] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:20.203791 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 21776
I20260924 23:11:20.204062 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:34271
--local_ip_for_outbound_sockets=127.12.212.1
--tserver_master_addrs=127.12.212.62:37739
--webserver_port=42375
--webserver_interface=127.12.212.1
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:34741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.212.1
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 23:11:20.205873 21844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 23:11:20.206199 21844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c: Bootstrap complete.
I20260924 23:11:20.208388 21844 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a015c65488448c93ea38671983817c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37739 } }
I20260924 23:11:20.208648 21844 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5a015c65488448c93ea38671983817c, State: Initialized, Role: FOLLOWER
I20260924 23:11:20.208828 21844 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a015c65488448c93ea38671983817c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37739 } }
I20260924 23:11:20.208910 21844 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:11:20.208961 21844 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:11:20.209007 21844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [term 1 FOLLOWER]: Advancing to term 2
I20260924 23:11:20.210083 21844 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a015c65488448c93ea38671983817c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37739 } }
I20260924 23:11:20.210193 21844 leader_election.cc:302] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5a015c65488448c93ea38671983817c; no voters:
I20260924 23:11:20.210376 21844 leader_election.cc:288] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 23:11:20.210467 21848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [term 2 FOLLOWER]: Leader election won for term 2
I20260924 23:11:20.210625 21848 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [term 2 LEADER]: Becoming Leader. State: Replica: c5a015c65488448c93ea38671983817c, State: Running, Role: LEADER
I20260924 23:11:20.210664 21844 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:11:20.210762 21848 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a015c65488448c93ea38671983817c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37739 } }
I20260924 23:11:20.211079 21849 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c5a015c65488448c93ea38671983817c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a015c65488448c93ea38671983817c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37739 } } }
I20260924 23:11:20.211164 21849 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:20.211421 21850 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5a015c65488448c93ea38671983817c. Latest consensus state: current_term: 2 leader_uuid: "c5a015c65488448c93ea38671983817c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a015c65488448c93ea38671983817c" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37739 } } }
I20260924 23:11:20.211542 21850 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:20.211594 21855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:11:20.212165 21855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:11:20.212314 21855 catalog_manager.cc:1259] Loaded cluster ID: 9493ed3a6b8740cdaa0bd7d9c77e29d6
I20260924 23:11:20.212364 21855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:11:20.213594 21855 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:11:20.213783 21855 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c5a015c65488448c93ea38671983817c: Loaded TSK: 0
I20260924 23:11:20.214004 21855 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 23:11:20.286496 21847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:20.286748 21847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:20.286787 21847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:20.288069 21847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:20.288115 21847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:20.288142 21847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:11:20.289662 21847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:34741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.1.keytab
--principal=oryx/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:34271
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=42375
--tserver_master_addrs=127.12.212.62:37739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:20.289915 21847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:20.290167 21847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:20.295581 21873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:20.295547 21870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:20.295773 21871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:20.295804 21847 server_base.cc:1034] running on GCE node
I20260924 23:11:20.296156 21847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:20.296473 21847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:20.297658 21847 hybrid_clock.cc:648] HybridClock initialized: now 1790291480297639 us; error 34 us; skew 500 ppm
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291480, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:20.300721 21847 init.cc:377] Logged in from keytab as oryx/127.12.212.1@KRBTEST.COM (short username oryx)
I20260924 23:11:20.301281 21847 webserver.cc:460] Webserver started at http://127.12.212.1:42375/ using document root <none> and password file <none>
I20260924 23:11:20.301563 21847 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:20.301637 21847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:20.303463 21847 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:11:20.304092 21880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:20.304253 21847 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:20.304358 21847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "d8ab630f4440400da09554be6062e132"
format_stamp: "Formatted at 2026-09-24 23:11:19 on dist-test-slave-m0z6"
server_key: "c04bd4af8d85f15aa1b5de0259aea950"
server_key_iv: "8d65b3f94b493cd238c55d32eb4a0f80"
server_key_version: "encryptionkey@0"
I20260924 23:11:20.304795 21847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:20.325059 21847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:20.327649 21847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:20.327797 21847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:20.328051 21847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:20.328385 21847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:20.328425 21847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:20.328462 21847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:20.328485 21847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:20.335279 21847 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:34271
I20260924 23:11:20.335314 21993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:34271 every 8 connection(s)
I20260924 23:11:20.335728 21847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:11:20.339318 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 21847
I20260924 23:11:20.339620 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:41513
--local_ip_for_outbound_sockets=127.12.212.2
--tserver_master_addrs=127.12.212.62:37739
--webserver_port=45381
--webserver_interface=127.12.212.2
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:34741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.212.2
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:20.344378 21996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:20.336409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.1:45059)
0924 23:11:20.336666 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:20.336712 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:20.337207 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:20.337631 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:20.337635 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:20.338203 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:20.338397 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:20.338402 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.339743 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.339744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:20.340898 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:20.340901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.341440 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.341441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:20.341564 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:20.342038 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:20.342047 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:20.344165 (+ 2118us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":168,"threads_started":1}
W20260924 23:11:20.344619 21994 heartbeater.cc:644] Failed to heartbeat to 127.12.212.62:37739 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.212.62:37739: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
W20260924 23:11:20.344748 21997 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:20.336513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:45059 (local address 127.12.212.62:37739)
0924 23:11:20.336701 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:20.336712 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:20.336800 (+ 88us) server_negotiation.cc:407] Connection header received
0924 23:11:20.337416 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:20.337422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:20.337466 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:20.337579 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:20.338520 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.339623 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.340971 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.341363 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.341457 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:20.344289 (+ 2832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.1:45059: BlockingRecv error: recv got EOF from 127.12.212.1:45059 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":80,"threads_started":1}
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:20.349107 21996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:20.344822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.1:51183)
0924 23:11:20.344983 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:20.344993 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:20.345034 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:20.345220 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:20.345221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:20.345365 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:20.345407 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:20.345409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.346004 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.346005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:20.346500 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:20.346501 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.346861 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.346862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:20.346923 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:20.347299 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:20.347304 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:20.349051 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":67,"threads_started":1}
W20260924 23:11:20.349241 21997 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:20.344988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:51183 (local address 127.12.212.62:37739)
0924 23:11:20.345122 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:20.345123 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:20.345129 (+ 6us) server_negotiation.cc:407] Connection header received
0924 23:11:20.345146 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:20.345147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:20.345157 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:20.345203 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:20.345473 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.345940 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.346569 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.346796 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.346846 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:20.349176 (+ 2330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.1:51183: BlockingRecv error: recv got EOF from 127.12.212.1:51183 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:20.353919 21996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:20.349497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.1:44497)
0924 23:11:20.349588 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:20.349597 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:20.349628 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:20.349746 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:20.349746 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:20.349881 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:20.349909 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:20.349911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.350826 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.350827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:20.351326 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:20.351327 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.351750 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.351751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:20.351785 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:20.352232 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:20.352236 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:20.353865 (+ 1629us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260924 23:11:20.354027 21997 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:20.349558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:44497 (local address 127.12.212.62:37739)
0924 23:11:20.349606 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:20.349608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:20.349619 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:11:20.349670 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:20.349671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:20.349681 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:20.349747 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:20.349972 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.350755 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.351397 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.351694 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.351753 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:20.353961 (+ 2208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.1:44497: BlockingRecv error: recv got EOF from 127.12.212.1:44497 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260924 23:11:20.354136 21994 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 23:11:20.421798 21998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:20.422065 21998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:20.422096 21998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:20.423346 21998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:20.423394 21998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:20.423430 21998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:11:20.424870 21998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:34741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.2.keytab
--principal=oryx/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:41513
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=45381
--tserver_master_addrs=127.12.212.62:37739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:20.425099 21998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:20.425304 21998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:20.430047 22008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:20.430018 22006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:20.430016 22005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:20.430047 21998 server_base.cc:1034] running on GCE node
I20260924 23:11:20.430439 21998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:20.430672 21998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:20.431833 21998 hybrid_clock.cc:648] HybridClock initialized: now 1790291480431823 us; error 27 us; skew 500 ppm
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291480, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:20.434069 21998 init.cc:377] Logged in from keytab as oryx/127.12.212.2@KRBTEST.COM (short username oryx)
I20260924 23:11:20.434475 21998 webserver.cc:460] Webserver started at http://127.12.212.2:45381/ using document root <none> and password file <none>
I20260924 23:11:20.434669 21998 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:20.434758 21998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:20.436064 21998 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:20.436581 22015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:20.436739 21998 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:20.436820 21998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "89b79c99b28e4dbf876574f8ea6f9e56"
format_stamp: "Formatted at 2026-09-24 23:11:19 on dist-test-slave-m0z6"
server_key: "451fd5f2fc5093a6adcfa94fdc4afa89"
server_key_iv: "4dcd97db3504bf4dc49bfdafeebe95b4"
server_key_version: "encryptionkey@0"
I20260924 23:11:20.437158 21998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:20.459968 21998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:20.461758 21998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:20.461902 21998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:20.462144 21998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:20.462443 21998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:20.462477 21998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:20.462507 21998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:20.462522 21998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:20.467999 21998 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:41513
I20260924 23:11:20.468050 22128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:41513 every 8 connection(s)
I20260924 23:11:20.468393 21998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:20.476169 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 21998
I20260924 23:11:20.476382 22131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:20.469106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.2:51305)
0924 23:11:20.469267 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:20.469312 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:20.469793 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:20.470069 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:20.470072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:20.470634 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:20.470941 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:20.470947 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.471670 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.471671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:20.472556 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:20.472559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.472973 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.472974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:20.473089 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:20.473588 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:20.473598 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:20.476168 (+ 2570us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
W20260924 23:11:20.476382 21997 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:20.469151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:51305 (local address 127.12.212.62:37739)
0924 23:11:20.469260 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:20.469262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:20.469453 (+ 191us) server_negotiation.cc:407] Connection header received
0924 23:11:20.469920 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:20.469921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:20.469937 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:20.470008 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:20.471059 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.471579 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.472664 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.472882 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.472946 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:20.476318 (+ 3372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.2:51305: BlockingRecv error: recv got EOF from 127.12.212.2:51305 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260924 23:11:20.476516 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:44665
--local_ip_for_outbound_sockets=127.12.212.3
--tserver_master_addrs=127.12.212.62:37739
--webserver_port=44199
--webserver_interface=127.12.212.3
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:34741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.212.3
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:20.476920 22129 heartbeater.cc:644] Failed to heartbeat to 127.12.212.62:37739 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.212.62:37739: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:20.482031 22131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:20.477202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.2:41171)
0924 23:11:20.477243 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:20.477254 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:20.477300 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:20.477426 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:20.477426 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:20.477574 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:20.477633 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:20.477635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.478538 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.478539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:20.479150 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:20.479153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.479613 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.479614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:20.479685 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:20.480131 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:20.480136 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:20.481978 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":21}
W20260924 23:11:20.482173 21997 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:20.477240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:41171 (local address 127.12.212.62:37739)
0924 23:11:20.477284 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:20.477287 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:20.477296 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:11:20.477366 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:20.477367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:20.477375 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:20.477419 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:20.477696 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.478452 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.479230 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.479502 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.479589 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:20.482097 (+ 2508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.2:41171: BlockingRecv error: recv got EOF from 127.12.212.2:41171 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:20.487084 22131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:20.482491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.2:51109)
0924 23:11:20.482560 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:20.482571 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:20.482631 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:20.482798 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:20.482799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:20.482913 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:20.482945 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:20.482946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.483827 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.483828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:20.484360 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:20.484363 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.484801 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.484802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:20.484844 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:20.485261 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:20.485267 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:20.487031 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260924 23:11:20.487226 22129 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 23:11:20.487356 21997 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:20.482565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:51109 (local address 127.12.212.62:37739)
0924 23:11:20.482624 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:20.482626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:20.482636 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:11:20.482710 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:20.482711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:20.482722 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:20.482774 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:20.483005 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.483732 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.484437 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.484722 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.484796 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:20.487128 (+ 2332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.2:51109: BlockingRecv error: recv got EOF from 127.12.212.2:51109 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260924 23:11:20.560348 22132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:20.560585 22132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:20.560609 22132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:20.561923 22132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:20.561973 22132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:20.562000 22132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:11:20.563585 22132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:34741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/krb5kdc/oryx_127.12.212.3.keytab
--principal=oryx/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:44665
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=44199
--tserver_master_addrs=127.12.212.62:37739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:20.563822 22132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:20.564043 22132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:11:20.569072 22132 server_base.cc:1034] running on GCE node
W20260924 23:11:20.569100 22137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:20.569144 22140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:20.569082 22138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:20.569463 22132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:20.569700 22132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:20.570844 22132 hybrid_clock.cc:648] HybridClock initialized: now 1790291480570831 us; error 32 us; skew 500 ppm
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291480, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:20.573357 22132 init.cc:377] Logged in from keytab as oryx/127.12.212.3@KRBTEST.COM (short username oryx)
I20260924 23:11:20.573837 22132 webserver.cc:460] Webserver started at http://127.12.212.3:44199/ using document root <none> and password file <none>
I20260924 23:11:20.574076 22132 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:20.574146 22132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:20.575732 22132 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:20.576534 22147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:20.576735 22132 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:20.576871 22132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "3c2b6d479c144b0caed21854f96d904e"
format_stamp: "Formatted at 2026-09-24 23:11:19 on dist-test-slave-m0z6"
server_key: "9b79ce323639acea2e35ce3ccc509ca0"
server_key_iv: "d9ab4ec8ebdaaaa50763dbf6ed8de13b"
server_key_version: "encryptionkey@0"
I20260924 23:11:20.577229 22132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:20.588351 22132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:20.590432 22132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:20.590570 22132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:20.590827 22132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:20.591120 22132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:20.591153 22132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:20.591183 22132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:20.591202 22132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:20.596622 22132 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:44665
I20260924 23:11:20.596665 22260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:44665 every 8 connection(s)
I20260924 23:11:20.597013 22132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:11:20.601907 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 22132
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
W20260924 23:11:20.604604 21997 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:20.597711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:33121 (local address 127.12.212.62:37739)
0924 23:11:20.597795 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:20.597797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:20.597993 (+ 196us) server_negotiation.cc:407] Connection header received
0924 23:11:20.598530 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:20.598531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:20.598542 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:20.598592 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:20.599584 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.600068 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.601067 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.601251 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.601301 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:20.604557 (+ 3256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.3:33121: BlockingRecv error: recv got EOF from 127.12.212.3:33121 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 23:11:20.604671 22263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:20.597691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.3:33121)
0924 23:11:20.597868 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:20.597909 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:20.598444 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:20.598650 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:20.598654 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:20.599296 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:20.599498 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:20.599504 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.600140 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.600141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:20.600986 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:20.600993 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.601343 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.601344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:20.601451 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:20.601992 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:20.602002 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:20.604468 (+ 2466us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
W20260924 23:11:20.604899 22261 heartbeater.cc:644] Failed to heartbeat to 127.12.212.62:37739 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.212.62:37739: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:20.609331 22263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:20.605107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.3:51219)
0924 23:11:20.605243 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:20.605252 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:20.605295 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:20.605411 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:20.605412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:20.605532 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:20.605575 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:20.605576 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.606140 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.606141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:20.606644 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:20.606645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.606964 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.606964 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:20.607025 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:20.607422 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:20.607427 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:20.609254 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":69,"threads_started":1}
W20260924 23:11:20.609477 21997 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:20.605157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:51219 (local address 127.12.212.62:37739)
0924 23:11:20.605199 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:20.605201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:20.605295 (+ 94us) server_negotiation.cc:407] Connection header received
0924 23:11:20.605348 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:20.605349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:20.605358 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:20.605404 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:20.605635 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.606077 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.606728 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.606909 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.606960 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:20.609418 (+ 2458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.3:51219: BlockingRecv error: recv got EOF from 127.12.212.3:51219 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 23:11:20.610085 22000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:20.602406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56916 (local address 127.12.212.62:37739)
0924 23:11:20.602467 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:20.602469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:20.602505 (+ 36us) server_negotiation.cc:407] Connection header received
0924 23:11:20.602560 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:20.602562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:20.602576 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:20.602650 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:20.603380 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.603979 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.605030 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.605308 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.605370 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:20.606519 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:20.606523 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:20.606523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:20.606536 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:20.608162 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:20.608581 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:20.608584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:20.608585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:20.608715 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:20.608962 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:20.608963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:20.608964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:20.609326 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:20.609523 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:20.609890 (+ 367us) server_negotiation.cc:299] Negotiation successful
0924 23:11:20.610022 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:20 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:20.614121 22263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:20.609690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.3:48115)
0924 23:11:20.609792 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:20.609804 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:20.609849 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:20.609961 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:20.609962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:20.610091 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:20.610372 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:20.610374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.611226 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.611227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:20.611827 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:20.611829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.612162 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.612163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:20.612215 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:20.612588 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:20.612593 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:20.614071 (+ 1478us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":84}
W20260924 23:11:20.614207 21997 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:20.609746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:48115 (local address 127.12.212.62:37739)
0924 23:11:20.609788 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:20.609790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:20.609851 (+ 61us) server_negotiation.cc:407] Connection header received
0924 23:11:20.609894 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:20.609894 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:20.609905 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:20.609957 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:20.610449 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.611163 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.611887 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:20.612106 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:20.612158 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:20.614155 (+ 1997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.3:48115: BlockingRecv error: recv got EOF from 127.12.212.3:48115 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 23:11:20.614255 22261 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 24 23:11:21 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:21 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:21.359408 22266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:21.354562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.1:37031)
0924 23:11:21.354750 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:21.354763 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:21.354823 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:21.355009 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:21.355010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:21.355173 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:21.355245 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:21.355246 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:21.355915 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:21.355917 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:21.356476 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:21.356478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:21.356833 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:21.356834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:21.356870 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:21.357372 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:21.357377 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:21.359336 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":65,"threads_started":1}
W20260924 23:11:21.359520 22267 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:21.354748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:37031 (local address 127.12.212.62:37739)
0924 23:11:21.354854 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:21.354856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:21.354864 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:11:21.354888 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:21.354889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:21.354898 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:21.354957 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:21.355319 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:21.355826 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:21.356564 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:21.356773 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:21.356826 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:21.359450 (+ 2624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.1:37031: BlockingRecv error: recv got EOF from 127.12.212.1:37031 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":46,"threads_started":1}
Sep 24 23:11:21 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:21 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:21.492086 22268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:21.487607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.2:57485)
0924 23:11:21.487744 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:21.487755 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:21.487827 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:21.487962 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:21.487963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:21.488131 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:21.488177 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:21.488179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:21.488824 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:21.488825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:21.489390 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:21.489392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:21.489808 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:21.489809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:21.489873 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:21.490299 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:21.490304 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:21.492016 (+ 1712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":69,"threads_started":1}
W20260924 23:11:21.492208 22267 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:21.487688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:57485 (local address 127.12.212.62:37739)
0924 23:11:21.487747 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:21.487749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:21.487816 (+ 67us) server_negotiation.cc:407] Connection header received
0924 23:11:21.487897 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:21.487898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:21.487910 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:21.487956 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:21.488244 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:21.488749 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:21.489465 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:21.489740 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:21.489790 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:21.492141 (+ 2351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.2:57485: BlockingRecv error: recv got EOF from 127.12.212.2:57485 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 24 23:11:21 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:21 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:21.619256 22269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:21.614711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.3:37737)
0924 23:11:21.614855 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:21.614865 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:21.614921 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:21.615066 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:21.615066 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:21.615221 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:21.615269 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:21.615270 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:21.615929 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:21.615930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:21.616632 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:21.616636 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:21.616964 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:21.616965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:21.617020 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:21.617418 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:21.617422 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:21.619161 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
W20260924 23:11:21.619323 22267 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:21.614846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:37737 (local address 127.12.212.62:37739)
0924 23:11:21.614906 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:21.614908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:21.614918 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:11:21.614996 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:21.614997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:21.615009 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:21.615051 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:21.615330 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:21.615863 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:21.616705 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:21.616902 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:21.616955 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:21.619267 (+ 2312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.3:37737: BlockingRecv error: recv got EOF from 127.12.212.3:37737 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 24 23:11:22 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:22 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:22.364696 22270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:22.359986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.1:57565)
0924 23:11:22.360116 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:22.360128 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:22.360236 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:22.360406 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:22.360407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:22.360563 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:22.360612 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:22.360613 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:22.361324 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:22.361325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:22.361902 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:22.361904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:22.362254 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:22.362255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:22.362299 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:22.362824 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:22.362829 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:22.364613 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
W20260924 23:11:22.364857 22271 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:22.360113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:57565 (local address 127.12.212.62:37739)
0924 23:11:22.360256 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:22.360258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:22.360269 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:11:22.360299 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:22.360299 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:22.360310 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:22.360386 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:22.360704 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:22.361225 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:22.361992 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:22.362188 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:22.362239 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:22.364775 (+ 2536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.1:57565: BlockingRecv error: recv got EOF from 127.12.212.1:57565 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
Sep 24 23:11:22 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:22 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:22.497198 22272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:22.492544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.2:48971)
0924 23:11:22.492683 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:22.492696 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:22.492825 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:22.492973 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:22.492974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:22.493146 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:22.493196 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:22.493198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:22.493954 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:22.493956 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:22.494492 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:22.494494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:22.494892 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:22.494893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:22.494948 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:22.495384 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:22.495389 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:22.497122 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
W20260924 23:11:22.497282 22271 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:22.492623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:48971 (local address 127.12.212.62:37739)
0924 23:11:22.492686 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:22.492688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:22.492813 (+ 125us) server_negotiation.cc:407] Connection header received
0924 23:11:22.492895 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:22.492896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:22.492910 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:22.492960 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:22.493300 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:22.493868 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:22.494573 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:22.494821 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:22.494884 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:22.497222 (+ 2338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.2:48971: BlockingRecv error: recv got EOF from 127.12.212.2:48971 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 24 23:11:22 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:22 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:22.624657 22273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:22.619801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.3:58133)
0924 23:11:22.619943 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:22.619954 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:22.620013 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:22.620160 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:22.620161 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:22.620333 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:22.620380 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:22.620382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:22.621038 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:22.621039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:22.621598 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:22.621600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:22.621929 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:22.621931 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:22.621987 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:22.622441 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:22.622445 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:22.624592 (+ 2147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":61,"threads_started":1}
W20260924 23:11:22.624788 22271 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:22.619929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:58133 (local address 127.12.212.62:37739)
0924 23:11:22.619993 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:22.619995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:22.620005 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:11:22.620093 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:22.620094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:22.620107 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:22.620148 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:22.620451 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:22.620961 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:22.621680 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:22.621867 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:22.621925 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:22.624721 (+ 2796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.3:58133: BlockingRecv error: recv got EOF from 127.12.212.3:58133 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 24 23:11:23 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:23 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:23.370555 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:23.365571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.1:37105)
0924 23:11:23.365749 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:23.365761 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:23.365830 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:23.365960 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:23.365960 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:23.366117 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:23.366168 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:23.366170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:23.366934 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:23.366936 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:23.367482 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:23.367485 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:23.367907 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:23.367909 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:23.367960 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:23.368374 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:23.368377 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:23.370476 (+ 2099us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":77,"threads_started":1}
W20260924 23:11:23.370630 22275 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:23.365724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:37105 (local address 127.12.212.62:37739)
0924 23:11:23.365837 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:23.365839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:23.365849 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:11:23.365876 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:23.365876 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:23.365886 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:23.365944 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:23.366295 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:23.366838 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:23.367581 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:23.367809 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:23.367869 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:23.370571 (+ 2702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.1:37105: BlockingRecv error: recv got EOF from 127.12.212.1:37105 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":48,"threads_started":1}
Sep 24 23:11:23 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:23 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:23.502470 22276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:23.497787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.2:37437)
0924 23:11:23.497906 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:23.497918 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:23.497979 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:23.498164 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:23.498165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:23.498328 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:23.498385 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:23.498386 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:23.499093 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:23.499094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:23.499633 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:23.499634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:23.500000 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:23.500000 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:23.500043 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:23.500491 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:23.500496 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:23.502380 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
W20260924 23:11:23.502564 22275 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:23.497842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:37437 (local address 127.12.212.62:37739)
0924 23:11:23.497917 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:23.497919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:23.497975 (+ 56us) server_negotiation.cc:407] Connection header received
0924 23:11:23.498026 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:23.498027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:23.498036 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:23.498097 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:23.498468 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:23.499019 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:23.499747 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:23.499939 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:23.499992 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:23.502484 (+ 2492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.2:37437: BlockingRecv error: recv got EOF from 127.12.212.2:37437 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 24 23:11:23 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:23 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:23.630348 22277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:23.625300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.3:58335)
0924 23:11:23.625494 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:23.625506 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:23.625626 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:23.625822 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:23.625824 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:23.625986 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:23.626034 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:23.626035 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:23.626781 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:23.626782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:23.627328 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:23.627330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:23.627704 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:23.627705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:23.627754 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:23.628273 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:23.628278 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:23.630266 (+ 1988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":119,"threads_started":1}
W20260924 23:11:23.630431 22275 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:23.625460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:58335 (local address 127.12.212.62:37739)
0924 23:11:23.625507 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:23.625509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:23.625562 (+ 53us) server_negotiation.cc:407] Connection header received
0924 23:11:23.625691 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:23.625692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:23.625703 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:23.625759 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:23.626163 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:23.626664 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:23.627430 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:23.627627 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:23.627687 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:23.630377 (+ 2690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.3:58335: BlockingRecv error: recv got EOF from 127.12.212.3:58335 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 24 23:11:24 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:24 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:24.375645 22278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:24.371040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.1:48467)
0924 23:11:24.371204 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:24.371216 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:24.371288 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:24.371432 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:24.371433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:24.371595 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:24.371644 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:24.371646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:24.372324 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:24.372325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:24.372883 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:24.372885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:24.373242 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:24.373242 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:24.373279 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:24.373774 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:24.373779 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:24.375567 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":68,"threads_started":1}
W20260924 23:11:24.375746 22279 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:24.371146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:48467 (local address 127.12.212.62:37739)
0924 23:11:24.371273 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:24.371275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:24.371285 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:11:24.371344 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:24.371344 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:24.371354 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:24.371411 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:24.371727 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:24.372238 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:24.372974 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:24.373172 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:24.373238 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:24.375667 (+ 2429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.1:48467: BlockingRecv error: recv got EOF from 127.12.212.1:48467 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":54,"threads_started":1}
Sep 24 23:11:24 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:24 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:24.507515 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:24.503022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.2:41309)
0924 23:11:24.503202 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:24.503216 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:24.503296 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:24.503439 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:24.503440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:24.503606 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:24.503693 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:24.503697 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:24.504367 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:24.504368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:24.504892 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:24.504893 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:24.505222 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:24.505223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:24.505290 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:24.505721 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:24.505726 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:24.507436 (+ 1710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":67,"threads_started":1}
W20260924 23:11:24.507591 22279 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:24.503118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:41309 (local address 127.12.212.62:37739)
0924 23:11:24.503209 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:24.503212 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:24.503283 (+ 71us) server_negotiation.cc:407] Connection header received
0924 23:11:24.503361 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:24.503362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:24.503374 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:24.503421 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:24.503772 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:24.504286 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:24.504966 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:24.505159 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:24.505214 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:24.507539 (+ 2325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.2:41309: BlockingRecv error: recv got EOF from 127.12.212.2:41309 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 24 23:11:24 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:24 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:24.641196 22281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:24.630988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.3:60581)
0924 23:11:24.631151 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:24.631163 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:24.631222 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:24.631439 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:24.631440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:24.631697 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:24.631755 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:24.631758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:24.632446 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:24.632448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:24.632979 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:24.632980 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:24.633290 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:24.633290 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:24.633336 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:24.639246 (+ 5910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:24.639252 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:24.641111 (+ 1859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":77,"threads_started":1}
W20260924 23:11:24.641281 22279 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:24.631103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:60581 (local address 127.12.212.62:37739)
0924 23:11:24.631169 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:24.631171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:24.631212 (+ 41us) server_negotiation.cc:407] Connection header received
0924 23:11:24.631329 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:24.631331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:24.631347 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:24.631420 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:24.631849 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:24.632361 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:24.633041 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:24.633228 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:24.633272 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:24.641220 (+ 7948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.3:60581: BlockingRecv error: recv got EOF from 127.12.212.3:60581 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 24 23:11:25 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:25 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:25.381000 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:25.376241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.1:38411)
0924 23:11:25.376364 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:25.376375 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:25.376435 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:25.376588 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:25.376588 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:25.376757 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:25.376807 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:25.376808 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:25.377439 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:25.377440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:25.378001 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:25.378003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:25.378354 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:25.378354 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:25.378391 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:25.378863 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:25.378868 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:25.380919 (+ 2051us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
W20260924 23:11:25.381052 22283 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:25.376335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:38411 (local address 127.12.212.62:37739)
0924 23:11:25.376448 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:25.376450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:25.376459 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:11:25.376484 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:25.376485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:25.376494 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:25.376553 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:25.376871 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:25.377367 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:25.378090 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:25.378296 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:25.378350 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:25.380993 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.1:38411: BlockingRecv error: recv got EOF from 127.12.212.1:38411 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
Sep 24 23:11:25 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:25 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:25.512993 22284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:25.508117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.2:44319)
0924 23:11:25.508273 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:25.508287 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:25.508356 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:25.508549 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:25.508551 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:25.508737 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:25.508783 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:25.508785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:25.509434 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:25.509434 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:25.510026 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:25.510029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:25.510395 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:25.510396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:25.510449 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:25.510922 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:25.510926 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:25.512906 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":87,"threads_started":1}
W20260924 23:11:25.513070 22283 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:25.508290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:44319 (local address 127.12.212.62:37739)
0924 23:11:25.508400 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:25.508402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:25.508414 (+ 12us) server_negotiation.cc:407] Connection header received
0924 23:11:25.508446 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:25.508447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:25.508458 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:25.508531 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:25.508861 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:25.509363 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:25.510117 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:25.510337 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:25.510396 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:25.513009 (+ 2613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.2:44319: BlockingRecv error: recv got EOF from 127.12.212.2:44319 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 24 23:11:25 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:25 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:25.647017 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:25.641720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.3:34923)
0924 23:11:25.641869 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:25.641880 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:25.641951 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:25.642134 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:25.642135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:25.642308 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:25.642357 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:25.642359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:25.643177 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:25.643178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:25.643735 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:25.643737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:25.644114 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:25.644115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:25.644162 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:25.644738 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:25.644743 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:25.646909 (+ 2166us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":61,"threads_started":1}
W20260924 23:11:25.647074 22283 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:25.641835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:34923 (local address 127.12.212.62:37739)
0924 23:11:25.641891 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:25.641892 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:25.641933 (+ 41us) server_negotiation.cc:407] Connection header received
0924 23:11:25.642002 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:25.642004 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:25.642014 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:25.642065 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:25.642538 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:25.643080 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:25.643822 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:25.644016 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:25.644109 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:25.647017 (+ 2908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.3:34923: BlockingRecv error: recv got EOF from 127.12.212.3:34923 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 24 23:11:26 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:26 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:26.386246 22286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:26.381506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.1:45237)
0924 23:11:26.381643 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:26.381656 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:26.381716 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:26.381865 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:26.381865 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:26.382019 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:26.382069 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:26.382070 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:26.382815 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:26.382818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:26.383369 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:26.383372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:26.383755 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:26.383756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:26.383806 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:26.384281 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:26.384286 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:26.386165 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
W20260924 23:11:26.386336 22287 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:26.381608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:45237 (local address 127.12.212.62:37739)
0924 23:11:26.381720 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:26.381721 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:26.381732 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:11:26.381784 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:26.381785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:26.381796 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:26.381851 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:26.382163 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:26.382731 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:26.383462 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:26.383676 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:26.383734 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:26.386275 (+ 2541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.1:45237: BlockingRecv error: recv got EOF from 127.12.212.1:45237 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":56,"threads_started":1}
Sep 24 23:11:26 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:26 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:26.518394 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:26.513509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.2:35519)
0924 23:11:26.513691 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:26.513703 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:26.513777 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:26.513939 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:26.513940 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:26.514112 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:26.514161 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:26.514163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:26.514961 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:26.514962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:26.515497 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:26.515499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:26.515835 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:26.515836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:26.515879 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:26.516331 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:26.516336 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:26.518314 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":81,"threads_started":1}
W20260924 23:11:26.518462 22287 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:26.513626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:35519 (local address 127.12.212.62:37739)
0924 23:11:26.513675 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:26.513677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:26.513820 (+ 143us) server_negotiation.cc:407] Connection header received
0924 23:11:26.513853 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:26.513854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:26.513868 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:26.513930 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:26.514256 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:26.514881 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:26.515587 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:26.515775 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:26.515824 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:26.518409 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.2:35519: BlockingRecv error: recv got EOF from 127.12.212.2:35519 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 24 23:11:26 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:26 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:26.652104 22289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:26.647555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.3:46223)
0924 23:11:26.647678 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:26.647690 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:26.647744 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:26.647922 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:26.647924 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:26.648090 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:26.648138 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:26.648139 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:26.648898 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:26.648899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:26.649456 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:26.649458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:26.649806 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:26.649806 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:26.649842 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:26.650263 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:26.650269 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:26.652043 (+ 1774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
W20260924 23:11:26.652199 22287 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:26.647657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:46223 (local address 127.12.212.62:37739)
0924 23:11:26.647735 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:26.647736 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:26.647749 (+ 13us) server_negotiation.cc:407] Connection header received
0924 23:11:26.647825 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:26.647826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:26.647839 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:26.647903 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:26.648223 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:26.648837 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:26.649532 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:26.649748 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:26.649801 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:26.652133 (+ 2332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.3:46223: BlockingRecv error: recv got EOF from 127.12.212.3:46223 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 24 23:11:27 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:27 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:27.391271 22290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:27.386806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.1:38367)
0924 23:11:27.386948 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:27.386962 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:27.387031 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:27.387177 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:27.387178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:27.387329 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:27.387377 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:27.387379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:27.387986 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:27.387987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:27.388523 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:27.388524 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:27.388847 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:27.388847 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:27.388886 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:27.389405 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:27.389410 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:27.391197 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
W20260924 23:11:27.391387 22291 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:27.386887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:38367 (local address 127.12.212.62:37739)
0924 23:11:27.387014 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:27.387016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:27.387025 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:11:27.387080 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:27.387080 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:27.387089 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:27.387166 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:27.387429 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:27.387919 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:27.388588 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:27.388790 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:27.388840 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:27.391317 (+ 2477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.1:38367: BlockingRecv error: recv got EOF from 127.12.212.1:38367 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
Sep 24 23:11:27 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:27 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:27.523780 22292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:27.518977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.2:42811)
0924 23:11:27.519130 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:27.519142 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:27.519195 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:27.519308 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:27.519309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:27.519463 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:27.519512 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:27.519514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:27.520225 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:27.520226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:27.520838 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:27.520841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:27.521213 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:27.521214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:27.521267 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:27.521774 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:27.521780 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:27.523702 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
W20260924 23:11:27.523873 22291 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:27.519136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:42811 (local address 127.12.212.62:37739)
0924 23:11:27.519189 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:27.519191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:27.519201 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:11:27.519230 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:27.519231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:27.519242 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:27.519298 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:27.519593 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:27.520115 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:27.520938 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:27.521139 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:27.521199 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:27.523818 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.2:42811: BlockingRecv error: recv got EOF from 127.12.212.2:42811 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 24 23:11:27 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:27 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:27.657245 22293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:27.652583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.3:33113)
0924 23:11:27.652775 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:27.652785 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:27.652842 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:27.652970 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:27.652970 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:27.653118 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:27.653193 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:27.653195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:27.653804 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:27.653805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:27.654352 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:27.654354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:27.654791 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:27.654792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:27.654842 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:27.655304 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:27.655309 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:27.657169 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":111,"threads_started":1}
W20260924 23:11:27.657321 22291 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:27.652676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:33113 (local address 127.12.212.62:37739)
0924 23:11:27.652731 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:27.652733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:27.652835 (+ 102us) server_negotiation.cc:407] Connection header received
0924 23:11:27.652903 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:27.652904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:27.652914 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:27.652953 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:27.653245 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:27.653734 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:27.654452 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:27.654664 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:27.654762 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:27.657268 (+ 2506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.3:33113: BlockingRecv error: recv got EOF from 127.12.212.3:33113 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Sep 24 23:11:28 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:28 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:28.396829 22294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:28.391909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.1:35971)
0924 23:11:28.392099 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:28.392114 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:28.392225 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:28.392413 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:28.392414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:28.392593 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:28.392655 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:28.392657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:28.393375 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:28.393376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:28.393907 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:28.393909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:28.394293 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:28.394294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:28.394347 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:28.394929 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:28.394933 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:28.396767 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":79,"threads_started":1}
W20260924 23:11:28.396946 22295 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:28.392081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:35971 (local address 127.12.212.62:37739)
0924 23:11:28.392214 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:28.392216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:28.392225 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:11:28.392276 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:28.392276 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:28.392293 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:28.392352 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:28.392770 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:28.393306 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:28.393997 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:28.394201 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:28.394266 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:28.396876 (+ 2610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.1:35971: BlockingRecv error: recv got EOF from 127.12.212.1:35971 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
Sep 24 23:11:28 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:28 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:28.529107 22296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:28.524372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.2:52927)
0924 23:11:28.524502 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:28.524512 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:28.524582 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:28.524750 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:28.524751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:28.524961 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:28.525012 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:28.525014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:28.525796 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:28.525797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:28.526329 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:28.526331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:28.526735 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:28.526737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:28.526790 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:28.527276 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:28.527281 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:28.529024 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
W20260924 23:11:28.529201 22295 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:28.524433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:52927 (local address 127.12.212.62:37739)
0924 23:11:28.524488 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:28.524490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:28.524574 (+ 84us) server_negotiation.cc:407] Connection header received
0924 23:11:28.524653 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:28.524654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:28.524669 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:28.524725 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:28.525074 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:28.525726 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:28.526396 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:28.526585 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:28.526638 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:28.529114 (+ 2476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.2:52927: BlockingRecv error: recv got EOF from 127.12.212.2:52927 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Sep 24 23:11:28 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:28 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:28.662334 22297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:28.657706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.3:35189)
0924 23:11:28.657899 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:28.657911 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:28.657977 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:28.658143 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:28.658144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:28.658312 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:28.658358 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:28.658360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:28.659072 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:28.659074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:28.659596 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:28.659598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:28.659907 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:28.659908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:28.659964 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:28.660434 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:28.660438 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:28.662246 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":114,"threads_started":1}
W20260924 23:11:28.662405 22295 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:28.657783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:35189 (local address 127.12.212.62:37739)
0924 23:11:28.657865 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:28.657867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:28.657966 (+ 99us) server_negotiation.cc:407] Connection header received
0924 23:11:28.658037 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:28.658037 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:28.658051 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:28.658127 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:28.658425 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:28.658992 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:28.659667 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:28.659850 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:28.659903 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:28.662349 (+ 2446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.3:35189: BlockingRecv error: recv got EOF from 127.12.212.3:35189 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 24 23:11:29 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:29 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:29.402019 22298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:29.397343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.1:53673)
0924 23:11:29.397481 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:29.397490 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:29.397535 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:29.397785 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:29.397786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:29.397961 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:29.398010 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:29.398012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:29.398738 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:29.398739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:29.399267 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:29.399269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:29.399647 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:29.399648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:29.399703 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:29.400187 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:29.400192 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:29.401946 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":62,"threads_started":1}
W20260924 23:11:29.402163 22299 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:29.397469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:53673 (local address 127.12.212.62:37739)
0924 23:11:29.397591 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:29.397593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:29.397601 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:11:29.397624 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:29.397625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:29.397636 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:29.397769 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:29.398071 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:29.398609 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:29.399343 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:29.399556 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:29.399608 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:29.402076 (+ 2468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.1:53673: BlockingRecv error: recv got EOF from 127.12.212.1:53673 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":67,"threads_started":1}
Sep 24 23:11:29 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:29 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:29.534258 22300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:29.529666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.2:43005)
0924 23:11:29.529787 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:29.529810 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:29.529887 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:29.530028 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:29.530029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:29.530200 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:29.530247 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:29.530248 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:29.530977 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:29.530978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:29.531497 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:29.531498 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:29.531835 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:29.531836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:29.531889 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:29.532335 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:29.532340 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:29.534193 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
W20260924 23:11:29.534358 22299 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:29.529774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:43005 (local address 127.12.212.62:37739)
0924 23:11:29.529888 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:29.529890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:29.529900 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:11:29.529951 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:29.529952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:29.529964 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:29.530012 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:29.530328 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:29.530892 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:29.531580 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:29.531776 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:29.531831 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:29.534297 (+ 2466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.2:43005: BlockingRecv error: recv got EOF from 127.12.212.2:43005 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Sep 24 23:11:29 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:29 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:29.670672 22301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:29.662953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.3:59355)
0924 23:11:29.663099 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:29.663115 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:29.663177 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:29.663286 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:29.663287 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:29.663441 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:29.663487 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:29.663489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:29.664147 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:29.664148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:29.664681 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:29.664683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:29.665076 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:29.665078 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:29.665134 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:29.668740 (+ 3606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:29.668746 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:29.670593 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
W20260924 23:11:29.670790 22299 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:29.663028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:59355 (local address 127.12.212.62:37739)
0924 23:11:29.663082 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:29.663084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:29.663187 (+ 103us) server_negotiation.cc:407] Connection header received
0924 23:11:29.663218 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:29.663218 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:29.663232 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:29.663275 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:29.663572 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:29.664062 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:29.664758 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:29.664977 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:29.665040 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:29.670733 (+ 5693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.3:59355: BlockingRecv error: recv got EOF from 127.12.212.3:59355 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 24 23:11:30 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:30 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:30.407091 22302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:30.402562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.1:42981)
0924 23:11:30.402680 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:30.402722 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:30.402780 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:30.402982 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:30.402983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:30.403191 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:30.403244 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:30.403246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:30.403928 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:30.403929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:30.404464 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:30.404466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:30.404825 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:30.404826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:30.404872 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:30.405284 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:30.405288 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:30.407031 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":67,"threads_started":1}
W20260924 23:11:30.407248 22303 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:30.402639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:42981 (local address 127.12.212.62:37739)
0924 23:11:30.402844 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:30.402846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:30.402853 (+ 7us) server_negotiation.cc:407] Connection header received
0924 23:11:30.402877 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:30.402877 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:30.402889 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:30.402951 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:30.403317 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:30.403837 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:30.404555 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:30.404754 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:30.404808 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:30.407145 (+ 2337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.1:42981: BlockingRecv error: recv got EOF from 127.12.212.1:42981 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":143,"threads_started":1}
Sep 24 23:11:30 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:30 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:30.539125 22304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:30.534807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.2:58429)
0924 23:11:30.534936 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:30.534947 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:30.535001 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:30.535108 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:30.535109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:30.535265 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:30.535317 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:30.535319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:30.535944 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:30.535945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:30.536472 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:30.536473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:30.536813 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:30.536814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:30.536856 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:30.537273 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:30.537277 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:30.539057 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":55,"threads_started":1}
W20260924 23:11:30.539220 22303 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:30.534883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:58429 (local address 127.12.212.62:37739)
0924 23:11:30.534947 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:30.534948 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:30.534986 (+ 38us) server_negotiation.cc:407] Connection header received
0924 23:11:30.535047 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:30.535047 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:30.535057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:30.535102 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:30.535375 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:30.535858 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:30.536546 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:30.536732 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:30.536784 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:30.539161 (+ 2377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.2:58429: BlockingRecv error: recv got EOF from 127.12.212.2:58429 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 24 23:11:30 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:30 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:30.675912 22305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:30.671310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.3:54397)
0924 23:11:30.671478 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:30.671489 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:30.671538 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:30.671661 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:30.671661 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:30.671819 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:30.671868 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:30.671869 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:30.672591 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:30.672592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:30.673128 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:30.673130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:30.673513 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:30.673515 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:30.673563 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:30.674029 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:30.674033 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:30.675838 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":79,"threads_started":1}
W20260924 23:11:30.676026 22303 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:30.671480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:54397 (local address 127.12.212.62:37739)
0924 23:11:30.671526 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:30.671528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:30.671537 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:11:30.671587 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:30.671588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:30.671597 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:30.671655 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:30.671969 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:30.672524 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:30.673229 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:30.673430 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:30.673491 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:30.675961 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.3:54397: BlockingRecv error: recv got EOF from 127.12.212.3:54397 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 24 23:11:31 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:31 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:31.412732 22306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:31.407668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.1:52861)
0924 23:11:31.407850 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:31.407864 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:31.407939 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:31.408147 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:31.408148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:31.408357 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:31.408414 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:31.408416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:31.409104 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:31.409105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:31.409699 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:31.409701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:31.410155 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:31.410157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:31.410226 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:31.410661 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:31.410665 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:31.412662 (+ 1997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":79,"threads_started":1}
W20260924 23:11:31.412859 22307 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:31.407813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:52861 (local address 127.12.212.62:37739)
0924 23:11:31.408018 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:31.408020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:31.408028 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:11:31.408053 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:31.408054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:31.408064 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:31.408132 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:31.408464 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:31.409011 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:31.409780 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:31.410045 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:31.410147 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:31.412786 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.1:52861: BlockingRecv error: recv got EOF from 127.12.212.1:52861 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":96,"threads_started":1}
Sep 24 23:11:31 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:31 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:31.544606 22308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:31.539706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.2:57275)
0924 23:11:31.539837 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:31.539851 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:31.539926 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:31.540146 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:31.540147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:31.540320 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:31.540365 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:31.540366 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:31.541067 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:31.541068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:31.541655 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:31.541657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:31.542010 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:31.542011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:31.542063 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:31.542558 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:31.542563 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:31.544516 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":58,"threads_started":1}
W20260924 23:11:31.544710 22307 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:31.539857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:57275 (local address 127.12.212.62:37739)
0924 23:11:31.539922 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:31.539924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:31.539934 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:11:31.540057 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:31.540059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:31.540072 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:31.540117 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:31.540448 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:31.540968 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:31.541752 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:31.541943 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:31.542010 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:31.544632 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.2:57275: BlockingRecv error: recv got EOF from 127.12.212.2:57275 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 24 23:11:31 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:31 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:31.681664 22309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:31.676529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.3:48187)
0924 23:11:31.676689 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:31.676701 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:31.676779 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:31.677016 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:31.677018 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:31.677184 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:31.677233 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:31.677235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:31.677949 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:31.677950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:31.678637 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:31.678641 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:31.679061 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:31.679062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:31.679122 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:31.679616 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:31.679622 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:31.681570 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":62,"threads_started":1}
W20260924 23:11:31.681732 22307 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:31.676659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:48187 (local address 127.12.212.62:37739)
0924 23:11:31.676720 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:31.676722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:31.676774 (+ 52us) server_negotiation.cc:407] Connection header received
0924 23:11:31.676834 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:31.676835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:31.676851 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:31.676979 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:31.677337 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:31.677843 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:31.678797 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:31.678988 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:31.679048 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:31.681677 (+ 2629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.3:48187: BlockingRecv error: recv got EOF from 127.12.212.3:48187 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 24 23:11:32 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:32 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:32.417583 22310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:32.413208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.1:43131)
0924 23:11:32.413332 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:32.413343 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:32.413401 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:32.413578 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:32.413579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:32.413752 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:32.413801 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:32.413802 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:32.414485 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:32.414487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:32.415094 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:32.415096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:32.415428 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:32.415428 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:32.415465 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:32.415905 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:32.415910 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:32.417515 (+ 1605us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
W20260924 23:11:32.417694 22311 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:32.413305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:43131 (local address 127.12.212.62:37739)
0924 23:11:32.413452 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:32.413454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:32.413462 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:11:32.413487 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:32.413488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:32.413498 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:32.413558 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:32.413894 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:32.414410 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:32.415176 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:32.415371 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:32.415425 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:32.417615 (+ 2190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.1:43131: BlockingRecv error: recv got EOF from 127.12.212.1:43131 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":52,"threads_started":1}
Sep 24 23:11:32 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:32 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:32.550091 22312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:32.545266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.2:57509)
0924 23:11:32.545466 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:32.545481 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:32.545554 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:32.545683 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:32.545684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:32.545856 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:32.545901 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:32.545902 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:32.546624 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:32.546625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:32.547225 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:32.547228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:32.547592 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:32.547593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:32.547648 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:32.548169 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:32.548174 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:32.550019 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":71,"threads_started":1}
W20260924 23:11:32.550196 22311 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:32.545427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:57509 (local address 127.12.212.62:37739)
0924 23:11:32.545498 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:32.545499 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:32.545538 (+ 39us) server_negotiation.cc:407] Connection header received
0924 23:11:32.545605 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:32.545606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:32.545616 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:32.545666 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:32.545983 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:32.546530 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:32.547307 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:32.547498 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:32.547554 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:32.550137 (+ 2583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.2:57509: BlockingRecv error: recv got EOF from 127.12.212.2:57509 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 24 23:11:32 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:32 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:32.687181 22313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:32.682183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.3:43767)
0924 23:11:32.682328 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:32.682339 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:32.682415 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:32.682588 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:32.682590 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:32.682823 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:32.682881 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:32.682883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:32.683676 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:32.683677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:32.684259 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:32.684262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:32.684669 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:32.684670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:32.684722 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:32.685279 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:32.685285 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:32.687110 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":66,"threads_started":1}
W20260924 23:11:32.687283 22311 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:32.682288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:43767 (local address 127.12.212.62:37739)
0924 23:11:32.682332 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:32.682334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:32.682479 (+ 145us) server_negotiation.cc:407] Connection header received
0924 23:11:32.682506 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:32.682507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:32.682521 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:32.682574 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:32.683019 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:32.683574 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:32.684356 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:32.684576 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:32.684639 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:32.687219 (+ 2580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.3:43767: BlockingRecv error: recv got EOF from 127.12.212.3:43767 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 24 23:11:33 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:33 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:33.423126 22314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:33.418144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.1:41821)
0924 23:11:33.418307 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:33.418320 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:33.418386 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:33.418634 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:33.418635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:33.418814 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:33.418862 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:33.418864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:33.419559 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:33.419560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:33.420177 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:33.420180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:33.420531 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:33.420532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:33.420584 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:33.421171 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:33.421176 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:33.423050 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
W20260924 23:11:33.423240 22315 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:33.418296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:41821 (local address 127.12.212.62:37739)
0924 23:11:33.418426 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:33.418428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:33.418439 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:11:33.418520 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:33.418521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:33.418536 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:33.418612 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:33.418931 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:33.419426 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:33.420259 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:33.420468 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:33.420525 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:33.423174 (+ 2649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.1:41821: BlockingRecv error: recv got EOF from 127.12.212.1:41821 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":55,"threads_started":1}
Sep 24 23:11:33 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:33 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:33.555626 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:33.550604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.2:54387)
0924 23:11:33.550799 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:33.550811 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:33.550876 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:33.551015 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:33.551016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:33.551199 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:33.551246 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:33.551248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:33.551936 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:33.551938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:33.552600 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:33.552604 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:33.553018 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:33.553019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:33.553081 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:33.553569 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:33.553573 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:33.555563 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":120,"threads_started":1}
W20260924 23:11:33.555784 22315 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:33.550724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:54387 (local address 127.12.212.62:37739)
0924 23:11:33.550771 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:33.550773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:33.550871 (+ 98us) server_negotiation.cc:407] Connection header received
0924 23:11:33.550931 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:33.550931 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:33.550943 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:33.551005 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:33.551331 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:33.551841 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:33.552726 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:33.552931 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:33.552989 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:33.555714 (+ 2725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.2:54387: BlockingRecv error: recv got EOF from 127.12.212.2:54387 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 24 23:11:33 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:33 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:33.692627 22317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:33.687772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.3:35337)
0924 23:11:33.687892 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:33.687904 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:33.687971 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:33.688131 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:33.688132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:33.688344 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:33.688391 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:33.688393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:33.689113 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:33.689114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:33.689652 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:33.689654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:33.690057 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:33.690058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:33.690120 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:33.690619 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:33.690624 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:33.692564 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
W20260924 23:11:33.692739 22315 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:33.687904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:35337 (local address 127.12.212.62:37739)
0924 23:11:33.687951 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:33.687953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:33.687966 (+ 13us) server_negotiation.cc:407] Connection header received
0924 23:11:33.688022 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:33.688022 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:33.688035 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:33.688086 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:33.688505 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:33.689012 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:33.689747 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:33.689937 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:33.690053 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:33.692675 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.3:35337: BlockingRecv error: recv got EOF from 127.12.212.3:35337 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 24 23:11:34 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:34 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:34.428439 22318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:34.423663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.1:35965)
0924 23:11:34.423807 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:34.423818 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:34.423869 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:34.424114 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:34.424115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:34.424295 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:34.424344 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:34.424345 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:34.425050 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:34.425051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:34.425640 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:34.425642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:34.425997 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:34.425998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:34.426044 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:34.426524 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:34.426529 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:34.428378 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":61,"threads_started":1}
W20260924 23:11:34.428553 22319 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:34.423767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:35965 (local address 127.12.212.62:37739)
0924 23:11:34.423893 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:34.423894 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:34.423902 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:11:34.423927 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:34.423927 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:34.423938 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:34.424004 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:34.424450 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:34.424964 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:34.425730 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:34.425932 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:34.425986 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:34.428483 (+ 2497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.1:35965: BlockingRecv error: recv got EOF from 127.12.212.1:35965 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
Sep 24 23:11:34 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:34 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:34.560765 22320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:34.556170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.2:45067)
0924 23:11:34.556336 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:34.556348 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:34.556408 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:34.556532 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:34.556533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:34.556703 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:34.556748 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:34.556750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:34.557455 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:34.557456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:34.558003 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:34.558005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:34.558355 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:34.558356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:34.558396 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:34.558946 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:34.558951 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:34.560672 (+ 1721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":59,"threads_started":1}
W20260924 23:11:34.560833 22319 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:34.556332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:45067 (local address 127.12.212.62:37739)
0924 23:11:34.556389 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:34.556391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:34.556401 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:11:34.556472 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:34.556473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:34.556485 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:34.556524 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:34.556847 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:34.557383 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:34.558080 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:34.558293 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:34.558348 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:34.560782 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.2:45067: BlockingRecv error: recv got EOF from 127.12.212.2:45067 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 24 23:11:34 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:34 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.3@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:34.702172 22321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:34.693173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.3:49781)
0924 23:11:34.693386 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:34.693399 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:34.693479 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:34.693637 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:34.693638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:34.693860 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:34.693922 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:34.693925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:34.694548 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:34.694548 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:34.695140 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:34.695143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:34.695505 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:34.695506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:34.695553 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:34.700392 (+ 4839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:34.700397 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:34.702100 (+ 1703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":122,"threads_started":1}
W20260924 23:11:34.702317 22319 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:34.693275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:49781 (local address 127.12.212.62:37739)
0924 23:11:34.693358 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:34.693361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:34.693518 (+ 157us) server_negotiation.cc:407] Connection header received
0924 23:11:34.693539 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:34.693540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:34.693552 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:34.693616 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:34.694000 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:34.694489 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:34.695248 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:34.695433 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:34.695487 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:34.702250 (+ 6763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.3:49781: BlockingRecv error: recv got EOF from 127.12.212.3:49781 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 24 23:11:35 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:35 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.1@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:35.434015 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:35.429068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.1:57093)
0924 23:11:35.429231 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:35.429241 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:35.429301 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:35.429467 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:35.429467 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:35.429633 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:35.429686 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:35.429688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:35.430404 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:35.430406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:35.431007 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:35.431009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:35.431446 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:35.431448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:35.431499 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:35.431943 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:35.431947 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:35.433922 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":72,"threads_started":1}
W20260924 23:11:35.434108 22323 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:35.429200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:57093 (local address 127.12.212.62:37739)
0924 23:11:35.429347 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:35.429349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:35.429357 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:11:35.429382 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:35.429383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:35.429394 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:35.429453 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:35.429754 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:35.430311 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:35.431089 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:35.431332 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:35.431389 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:35.434037 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.1:57093: BlockingRecv error: recv got EOF from 127.12.212.1:57093 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":56,"threads_started":1}
Sep 24 23:11:35 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
Sep 24 23:11:35 dist-test-slave-m0z6 krb5kdc[21263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.212.2@KRBTEST.COM for oryx/127.12.212.62@KRBTEST.COM, Server not found in Kerberos database
I20260924 23:11:35.566263 22324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:35.561456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37739 (local address 127.12.212.2:56483)
0924 23:11:35.561611 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:35.561622 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:35.561687 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:35.561813 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:35.561813 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:35.561974 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:35.562025 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:35.562027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:35.562680 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:35.562681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:35.563354 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:35.563357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:35.563769 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:35.563770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:35.563813 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:35.564420 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:35.564425 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:35.566185 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.212.62:37739: Server oryx/127.12.212.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":63,"threads_started":1}
W20260924 23:11:35.566406 22323 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:35.561608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:56483 (local address 127.12.212.62:37739)
0924 23:11:35.561675 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:35.561677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:35.561687 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:11:35.561737 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:35.561738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:35.561748 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:35.561819 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:35.562090 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:35.562598 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:35.563431 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:35.563703 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:35.563764 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:35.566348 (+ 2584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.212.2:56483: BlockingRecv error: recv got EOF from 127.12.212.2:56483 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 23:11:35.606868 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 21847
I20260924 23:11:35.610855 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 21998
I20260924 23:11:35.614882 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 22132
I20260924 23:11:35.618585 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 21776
2026-09-24T23:11:35Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17365 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-24T23:11:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:11:35Z Disabled control of system clock
I20260924 23:11:35.644959 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:34685
--webserver_interface=127.12.212.62
--webserver_port=0
--builtin_ntp_servers=127.12.212.20:45403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:34685
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260924 23:11:35.724889 22328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:35.725107 22328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:35.725127 22328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:35.726368 22328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:11:35.726413 22328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:35.726423 22328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:35.726434 22328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:11:35.726445 22328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:11:35.727864 22328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:45403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:34685
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:34685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:35.728099 22328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:35.728379 22328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:35.733476 22336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:35.733500 22328 server_base.cc:1034] running on GCE node
W20260924 23:11:35.733467 22334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:35.733464 22333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:35.733932 22328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:35.734220 22328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:35.735368 22328 hybrid_clock.cc:648] HybridClock initialized: now 1790291495735331 us; error 47 us; skew 500 ppm
I20260924 23:11:35.736640 22328 webserver.cc:460] Webserver started at http://127.12.212.62:42283/ using document root <none> and password file <none>
I20260924 23:11:35.736888 22328 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:35.736938 22328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:35.737076 22328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:35.738162 22328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "1b910378e598432dbdc43b5dfa7a6953"
format_stamp: "Formatted at 2026-09-24 23:11:35 on dist-test-slave-m0z6"
server_key: "987a8e86cc6a1f593d3b1d07910185d7"
server_key_iv: "1ee5a69a0c050d29edfe927e60abf78a"
server_key_version: "encryptionkey@0"
I20260924 23:11:35.738487 22328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "1b910378e598432dbdc43b5dfa7a6953"
format_stamp: "Formatted at 2026-09-24 23:11:35 on dist-test-slave-m0z6"
server_key: "987a8e86cc6a1f593d3b1d07910185d7"
server_key_iv: "1ee5a69a0c050d29edfe927e60abf78a"
server_key_version: "encryptionkey@0"
I20260924 23:11:35.739778 22328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:11:35.740490 22342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:35.740657 22328 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 23:11:35.740741 22328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "1b910378e598432dbdc43b5dfa7a6953"
format_stamp: "Formatted at 2026-09-24 23:11:35 on dist-test-slave-m0z6"
server_key: "987a8e86cc6a1f593d3b1d07910185d7"
server_key_iv: "1ee5a69a0c050d29edfe927e60abf78a"
server_key_version: "encryptionkey@0"
I20260924 23:11:35.740824 22328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:35.774441 22328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:35.774814 22328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:35.774946 22328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:35.778529 22328 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:34685
I20260924 23:11:35.778589 22394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:34685 every 8 connection(s)
I20260924 23:11:35.778927 22328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:11:35.779529 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 22328
I20260924 23:11:35.779629 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:11:35.779688 22395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:35.779835 13136 external_mini_cluster.cc:1428] Setting key b250a4ace64035731711372dbb2baffd
I20260924 23:11:35.783557 22395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953: Bootstrap starting.
I20260924 23:11:35.784210 22395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:35.784529 22395 log.cc:824] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:35.785403 22395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953: No bootstrap required, opened a new log
I20260924 23:11:35.786221 22398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:35.780432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41758 (local address 127.12.212.62:34685)
0924 23:11:35.780606 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:35.780615 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:35.780652 (+ 37us) server_negotiation.cc:407] Connection header received
0924 23:11:35.781231 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:35.781242 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:35.781345 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:35.781534 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:35.781784 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:35.783156 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:35.784021 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:35.784636 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:35.784765 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:35.785592 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:35.785597 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:35.785597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:35.785790 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:35.785823 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:35.785835 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:35.785888 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:35.785973 (+ 85us) server_negotiation.cc:299] Negotiation successful
0924 23:11:35.786056 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
I20260924 23:11:35.787459 22395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b910378e598432dbdc43b5dfa7a6953" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34685 } }
I20260924 23:11:35.787599 22395 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:35.787622 22395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b910378e598432dbdc43b5dfa7a6953, State: Initialized, Role: FOLLOWER
I20260924 23:11:35.787714 22395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b910378e598432dbdc43b5dfa7a6953" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34685 } }
I20260924 23:11:35.787776 22395 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:11:35.787804 22395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:11:35.787837 22395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:35.788345 22395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b910378e598432dbdc43b5dfa7a6953" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34685 } }
I20260924 23:11:35.788436 22395 leader_election.cc:302] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b910378e598432dbdc43b5dfa7a6953; no voters:
I20260924 23:11:35.788576 22395 leader_election.cc:288] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:11:35.788652 22400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:35.788810 22400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953 [term 1 LEADER]: Becoming Leader. State: Replica: 1b910378e598432dbdc43b5dfa7a6953, State: Running, Role: LEADER
I20260924 23:11:35.788816 22395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:11:35.788920 22400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b910378e598432dbdc43b5dfa7a6953" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34685 } }
I20260924 23:11:35.789283 22402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b910378e598432dbdc43b5dfa7a6953. Latest consensus state: current_term: 1 leader_uuid: "1b910378e598432dbdc43b5dfa7a6953" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b910378e598432dbdc43b5dfa7a6953" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34685 } } }
I20260924 23:11:35.789304 22401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b910378e598432dbdc43b5dfa7a6953" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b910378e598432dbdc43b5dfa7a6953" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34685 } } }
I20260924 23:11:35.789448 22402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:35.789465 22401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953 [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:35.789674 22409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:11:35.790143 22409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:11:35.791746 22409 catalog_manager.cc:1347] Generated new cluster ID: 7055fbdf101d4611a10704f2d166b9cb
I20260924 23:11:35.791821 22409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:11:35.809465 22409 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:11:35.810026 22409 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:11:35.813807 22409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b910378e598432dbdc43b5dfa7a6953: Generated new TSK 0
I20260924 23:11:35.814031 22409 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:11:35.816781 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34685
--builtin_ntp_servers=127.12.212.20:45403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260924 23:11:35.914813 22419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:35.915035 22419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:35.915056 22419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:35.916371 22419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:35.916433 22419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:35.916460 22419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:11:35.917912 22419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:45403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:35.918175 22419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:35.918484 22419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:35.924398 22424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:35.924614 22419 server_base.cc:1034] running on GCE node
W20260924 23:11:35.924403 22425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:35.924436 22427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:35.925024 22419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:35.925309 22419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:35.926491 22419 hybrid_clock.cc:648] HybridClock initialized: now 1790291495926477 us; error 28 us; skew 500 ppm
I20260924 23:11:35.928051 22419 webserver.cc:460] Webserver started at http://127.12.212.1:41265/ using document root <none> and password file <none>
I20260924 23:11:35.928332 22419 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:35.928395 22419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:35.928521 22419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:35.930034 22419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "87403efd270b47879d7696f494b10931"
format_stamp: "Formatted at 2026-09-24 23:11:35 on dist-test-slave-m0z6"
server_key: "3ef19aa50238fa7e0af68a6f92e94bb8"
server_key_iv: "bdd99c265fc51e36fd53dbd407d2f527"
server_key_version: "encryptionkey@0"
I20260924 23:11:35.930440 22419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "87403efd270b47879d7696f494b10931"
format_stamp: "Formatted at 2026-09-24 23:11:35 on dist-test-slave-m0z6"
server_key: "3ef19aa50238fa7e0af68a6f92e94bb8"
server_key_iv: "bdd99c265fc51e36fd53dbd407d2f527"
server_key_version: "encryptionkey@0"
I20260924 23:11:35.932051 22419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260924 23:11:35.932874 22433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:35.933029 22419 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 23:11:35.933127 22419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "87403efd270b47879d7696f494b10931"
format_stamp: "Formatted at 2026-09-24 23:11:35 on dist-test-slave-m0z6"
server_key: "3ef19aa50238fa7e0af68a6f92e94bb8"
server_key_iv: "bdd99c265fc51e36fd53dbd407d2f527"
server_key_version: "encryptionkey@0"
I20260924 23:11:35.933198 22419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:35.955480 22419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:35.955803 22419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:35.955920 22419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:35.956100 22419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:35.956399 22419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:35.956430 22419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:35.956452 22419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:35.956470 22419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:35.961827 22419 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:32893
I20260924 23:11:35.961901 22546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:32893 every 8 connection(s)
I20260924 23:11:35.962203 22419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:11:35.962459 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 22419
I20260924 23:11:35.962560 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:11:35.962778 13136 external_mini_cluster.cc:1428] Setting key 14dbb08f2812d05420dca045b8c36192
I20260924 23:11:35.963748 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34685
--builtin_ntp_servers=127.12.212.20:45403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260924 23:11:35.967574 22398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:35.962988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:47611 (local address 127.12.212.62:34685)
0924 23:11:35.963061 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:35.963064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:35.963362 (+ 298us) server_negotiation.cc:407] Connection header received
0924 23:11:35.963962 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:35.963963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:35.963978 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:35.964007 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:35.964745 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:35.965241 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:35.966216 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:35.966413 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:35.966467 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:35.967295 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:35.967297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:35.967297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:35.967302 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:35.967317 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:35.967321 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:35.967365 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:35.967450 (+ 85us) server_negotiation.cc:299] Negotiation successful
0924 23:11:35.967490 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 23:11:35.967768 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:35.962998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:34685 (local address 127.12.212.1:47611)
0924 23:11:35.963224 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:35.963268 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:35.963825 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:35.964153 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:35.964156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:35.964165 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:35.964647 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:35.964651 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:35.965331 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:35.965332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:35.966131 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:35.966134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:35.966486 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:35.966487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:35.966619 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:35.967159 (+ 540us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:11:35.967159 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:35.967189 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:11:35.967213 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:35.967375 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:35.967380 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:35.967384 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:11:35.967448 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 23:11:35.967555 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260924 23:11:35.968209 22547 heartbeater.cc:347] Connected to a master server at 127.12.212.62:34685
I20260924 23:11:35.968295 22547 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:35.968469 22547 heartbeater.cc:511] Master 127.12.212.62:34685 requested a full tablet report, sending...
I20260924 23:11:35.968962 22359 ts_manager.cc:194] Registered new tserver with Master: 87403efd270b47879d7696f494b10931 (127.12.212.1:32893)
I20260924 23:11:35.969555 22359 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.212.1:47611
W20260924 23:11:36.047442 22550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:36.047655 22550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:36.047685 22550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:36.048914 22550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:36.048959 22550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:36.048993 22550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:11:36.050414 22550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:45403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:36.050673 22550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:36.051038 22550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:36.055745 22555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:36.055799 22556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:36.055922 22558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:36.056211 22550 server_base.cc:1034] running on GCE node
I20260924 23:11:36.056454 22550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:36.056689 22550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:36.057826 22550 hybrid_clock.cc:648] HybridClock initialized: now 1790291496057810 us; error 32 us; skew 500 ppm
I20260924 23:11:36.059103 22550 webserver.cc:460] Webserver started at http://127.12.212.2:34817/ using document root <none> and password file <none>
I20260924 23:11:36.059345 22550 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:36.059396 22550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:36.059527 22550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:36.060631 22550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "704a2e0559ea4cab84d3af946e5c1182"
format_stamp: "Formatted at 2026-09-24 23:11:36 on dist-test-slave-m0z6"
server_key: "1dce45a54eb0cf0ba6b0fee7a0e5896a"
server_key_iv: "5dd6e5c87cfd5f3f69e90b5c48b89bdd"
server_key_version: "encryptionkey@0"
I20260924 23:11:36.060998 22550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "704a2e0559ea4cab84d3af946e5c1182"
format_stamp: "Formatted at 2026-09-24 23:11:36 on dist-test-slave-m0z6"
server_key: "1dce45a54eb0cf0ba6b0fee7a0e5896a"
server_key_iv: "5dd6e5c87cfd5f3f69e90b5c48b89bdd"
server_key_version: "encryptionkey@0"
I20260924 23:11:36.062232 22550 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:11:36.063022 22564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:36.063179 22550 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:36.063269 22550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "704a2e0559ea4cab84d3af946e5c1182"
format_stamp: "Formatted at 2026-09-24 23:11:36 on dist-test-slave-m0z6"
server_key: "1dce45a54eb0cf0ba6b0fee7a0e5896a"
server_key_iv: "5dd6e5c87cfd5f3f69e90b5c48b89bdd"
server_key_version: "encryptionkey@0"
I20260924 23:11:36.063366 22550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:36.071928 22550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:36.072228 22550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:36.072359 22550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:36.072561 22550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:36.072863 22550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:36.072894 22550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:36.072922 22550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:36.072947 22550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:36.078522 22550 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:40105
I20260924 23:11:36.078570 22677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:40105 every 8 connection(s)
I20260924 23:11:36.078922 22550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:11:36.083995 22398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:36.079683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:54885 (local address 127.12.212.62:34685)
0924 23:11:36.079780 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:36.079782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:36.079935 (+ 153us) server_negotiation.cc:407] Connection header received
0924 23:11:36.080444 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:36.080446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:36.080455 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:36.080508 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:36.081112 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:36.081608 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:36.082664 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:36.082911 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:36.082972 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:36.083744 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:36.083745 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:36.083745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:36.083749 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:36.083762 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:36.083766 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:36.083825 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:36.083910 (+ 85us) server_negotiation.cc:299] Negotiation successful
0924 23:11:36.083933 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260924 23:11:36.084138 22680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:36.079627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:34685 (local address 127.12.212.2:54885)
0924 23:11:36.079813 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:36.079852 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:36.080351 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:36.080564 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:36.080567 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:36.080576 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:36.081039 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:36.081043 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:36.081703 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:36.081705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:36.082565 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:36.082568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:36.082986 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:36.082987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:36.083099 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:36.083624 (+ 525us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:11:36.083624 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:36.083653 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:11:36.083675 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:36.083826 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:36.083829 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:36.083831 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 23:11:36.083878 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 23:11:36.083959 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
I20260924 23:11:36.084589 22678 heartbeater.cc:347] Connected to a master server at 127.12.212.62:34685
I20260924 23:11:36.084681 22678 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:36.084864 22678 heartbeater.cc:511] Master 127.12.212.62:34685 requested a full tablet report, sending...
I20260924 23:11:36.085193 22359 ts_manager.cc:194] Registered new tserver with Master: 704a2e0559ea4cab84d3af946e5c1182 (127.12.212.2:40105)
I20260924 23:11:36.085531 22359 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.212.2:54885
I20260924 23:11:36.088420 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 22550
I20260924 23:11:36.088512 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:11:36.088678 13136 external_mini_cluster.cc:1428] Setting key 37e46f8f649ae5218c9ad4cd8acfa340
I20260924 23:11:36.089546 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34685
--builtin_ntp_servers=127.12.212.20:45403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260924 23:11:36.168879 22681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:36.169072 22681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:36.169091 22681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:36.170156 22681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:36.170185 22681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:36.170208 22681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:11:36.171474 22681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:45403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:36.171666 22681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:36.171924 22681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:36.176857 22687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:36.176930 22681 server_base.cc:1034] running on GCE node
W20260924 23:11:36.176860 22686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:36.176878 22689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:36.177302 22681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:36.177634 22681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:36.178800 22681 hybrid_clock.cc:648] HybridClock initialized: now 1790291496178775 us; error 44 us; skew 500 ppm
I20260924 23:11:36.180307 22681 webserver.cc:460] Webserver started at http://127.12.212.3:42927/ using document root <none> and password file <none>
I20260924 23:11:36.180554 22681 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:36.180608 22681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:36.180748 22681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:36.181870 22681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "cd0e893f8c8d401f9113771561de5078"
format_stamp: "Formatted at 2026-09-24 23:11:36 on dist-test-slave-m0z6"
server_key: "33d551c854af568315ab237bddafb36f"
server_key_iv: "eedb28ea76d9fc83a1ecc47bf4953e46"
server_key_version: "encryptionkey@0"
I20260924 23:11:36.182220 22681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "cd0e893f8c8d401f9113771561de5078"
format_stamp: "Formatted at 2026-09-24 23:11:36 on dist-test-slave-m0z6"
server_key: "33d551c854af568315ab237bddafb36f"
server_key_iv: "eedb28ea76d9fc83a1ecc47bf4953e46"
server_key_version: "encryptionkey@0"
I20260924 23:11:36.183665 22681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:11:36.184427 22695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:36.184593 22681 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:36.184677 22681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "cd0e893f8c8d401f9113771561de5078"
format_stamp: "Formatted at 2026-09-24 23:11:36 on dist-test-slave-m0z6"
server_key: "33d551c854af568315ab237bddafb36f"
server_key_iv: "eedb28ea76d9fc83a1ecc47bf4953e46"
server_key_version: "encryptionkey@0"
I20260924 23:11:36.184749 22681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:36.197381 22681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:36.197695 22681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:36.197841 22681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:36.198055 22681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:36.198400 22681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:36.198441 22681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:36.198472 22681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:36.198488 22681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:36.204058 22681 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:45163
I20260924 23:11:36.204124 22808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:45163 every 8 connection(s)
I20260924 23:11:36.204444 22681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:11:36.209652 22398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:36.205159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:59017 (local address 127.12.212.62:34685)
0924 23:11:36.205300 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:36.205302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:36.205507 (+ 205us) server_negotiation.cc:407] Connection header received
0924 23:11:36.205990 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:36.205991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:36.206002 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:36.206063 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:36.206752 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:36.207306 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:36.208316 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:36.208505 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:36.208566 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:36.209373 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:36.209374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:36.209375 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:36.209379 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:36.209396 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:36.209400 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:36.209452 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:36.209559 (+ 107us) server_negotiation.cc:299] Negotiation successful
0924 23:11:36.209585 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260924 23:11:36.209782 22811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:36.205138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:34685 (local address 127.12.212.3:59017)
0924 23:11:36.205319 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:36.205361 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:36.205881 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:36.206118 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:36.206122 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:36.206130 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:36.206642 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:36.206646 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:36.207423 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:36.207423 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:36.208225 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:36.208226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:36.208574 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:36.208574 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:36.208683 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:36.209214 (+ 531us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:11:36.209215 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:36.209248 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:11:36.209273 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:36.209460 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:36.209464 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:36.209466 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 23:11:36.209518 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 23:11:36.209607 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260924 23:11:36.210222 22809 heartbeater.cc:347] Connected to a master server at 127.12.212.62:34685
I20260924 23:11:36.210318 22809 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:36.210497 22809 heartbeater.cc:511] Master 127.12.212.62:34685 requested a full tablet report, sending...
I20260924 23:11:36.210886 22359 ts_manager.cc:194] Registered new tserver with Master: cd0e893f8c8d401f9113771561de5078 (127.12.212.3:45163)
I20260924 23:11:36.211251 22359 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.212.3:59017
I20260924 23:11:36.214174 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 22681
I20260924 23:11:36.214310 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:11:36.214484 13136 external_mini_cluster.cc:1428] Setting key 19ff7be27e857ca93f810951f7859945
I20260924 23:11:36.215340 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 23:11:36.216969 22398 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:36.216556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41760 (local address 127.12.212.62:34685)
0924 23:11:36.216610 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:36.216611 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:36.216711 (+ 100us) server_negotiation.cc:407] Connection header received
0924 23:11:36.216753 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:36.216754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:36.216762 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:36.216812 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:36.216919 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41760: BlockingRecv error: recv got EOF from 127.0.0.1:41760 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 23:11:36.217082 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 22419
I20260924 23:11:36.221298 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 22550
I20260924 23:11:36.225907 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 22681
I20260924 23:11:36.229557 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 22328
2026-09-24T23:11:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (609 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-24T23:11:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:11:36Z Disabled control of system clock
I20260924 23:11:36.254184 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:34787
--webserver_interface=127.12.212.62
--webserver_port=0
--builtin_ntp_servers=127.12.212.20:41835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:34787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 23:11:36.334223 22820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:36.334445 22820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:36.334467 22820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:36.335603 22820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:11:36.335637 22820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:36.335649 22820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:36.335659 22820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:11:36.335669 22820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:11:36.337152 22820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:41835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:34787
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:34787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:36.337414 22820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:36.337685 22820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:11:36.342582 22820 server_base.cc:1034] running on GCE node
W20260924 23:11:36.342548 22826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:36.342550 22825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:36.342550 22828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:36.343083 22820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:36.343444 22820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:36.344599 22820 hybrid_clock.cc:648] HybridClock initialized: now 1790291496344579 us; error 33 us; skew 500 ppm
I20260924 23:11:36.350508 22820 webserver.cc:460] Webserver started at http://127.12.212.62:35699/ using document root <none> and password file <none>
I20260924 23:11:36.350786 22820 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:36.350843 22820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:36.350975 22820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:36.352073 22820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "ae61b348556e44df9f149df9df2b921b"
format_stamp: "Formatted at 2026-09-24 23:11:36 on dist-test-slave-m0z6"
server_key: "30b948d6305bc6329e32d4ad5e5e8ddf"
server_key_iv: "ea25b36c563c887cc0e84ebe200b8b88"
server_key_version: "encryptionkey@0"
I20260924 23:11:36.352401 22820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "ae61b348556e44df9f149df9df2b921b"
format_stamp: "Formatted at 2026-09-24 23:11:36 on dist-test-slave-m0z6"
server_key: "30b948d6305bc6329e32d4ad5e5e8ddf"
server_key_iv: "ea25b36c563c887cc0e84ebe200b8b88"
server_key_version: "encryptionkey@0"
I20260924 23:11:36.353677 22820 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:11:36.354411 22834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:36.354576 22820 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 23:11:36.354658 22820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "ae61b348556e44df9f149df9df2b921b"
format_stamp: "Formatted at 2026-09-24 23:11:36 on dist-test-slave-m0z6"
server_key: "30b948d6305bc6329e32d4ad5e5e8ddf"
server_key_iv: "ea25b36c563c887cc0e84ebe200b8b88"
server_key_version: "encryptionkey@0"
I20260924 23:11:36.354753 22820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:36.358141 22820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:36.358412 22820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:36.358561 22820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:36.362252 22820 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:34787
I20260924 23:11:36.362285 22886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:34787 every 8 connection(s)
I20260924 23:11:36.362623 22820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:11:36.363350 22887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:36.366139 22887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b: Bootstrap starting.
I20260924 23:11:36.366750 22887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:36.367048 22887 log.cc:824] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:36.367700 22887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b: No bootstrap required, opened a new log
I20260924 23:11:36.368397 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 22820
I20260924 23:11:36.368475 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:11:36.368634 13136 external_mini_cluster.cc:1428] Setting key 1a9362fc1a71ec18b418fe877474a7f5
I20260924 23:11:36.369652 22887 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae61b348556e44df9f149df9df2b921b" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34787 } }
I20260924 23:11:36.369845 22887 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:36.369884 22887 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae61b348556e44df9f149df9df2b921b, State: Initialized, Role: FOLLOWER
I20260924 23:11:36.370008 22887 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae61b348556e44df9f149df9df2b921b" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34787 } }
I20260924 23:11:36.370082 22887 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:11:36.370110 22887 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:11:36.370146 22887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:36.370738 22887 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae61b348556e44df9f149df9df2b921b" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34787 } }
I20260924 23:11:36.370857 22887 leader_election.cc:302] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae61b348556e44df9f149df9df2b921b; no voters:
I20260924 23:11:36.371004 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:36.369114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39522 (local address 127.12.212.62:34787)
0924 23:11:36.369355 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:36.369366 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:36.369402 (+ 36us) server_negotiation.cc:407] Connection header received
0924 23:11:36.369619 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:36.369627 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:36.369677 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:36.369829 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:36.370531 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:36.370535 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:36.370536 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:36.370540 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:36.370579 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:36.370591 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:36.370648 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:36.370744 (+ 96us) server_negotiation.cc:299] Negotiation successful
0924 23:11:36.370802 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":78,"threads_started":1}
I20260924 23:11:36.371016 22887 leader_election.cc:288] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:11:36.371287 22892 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:36.371320 22887 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:11:36.371426 22892 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b [term 1 LEADER]: Becoming Leader. State: Replica: ae61b348556e44df9f149df9df2b921b, State: Running, Role: LEADER
I20260924 23:11:36.371520 22892 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae61b348556e44df9f149df9df2b921b" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34787 } }
I20260924 23:11:36.371825 22892 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae61b348556e44df9f149df9df2b921b. Latest consensus state: current_term: 1 leader_uuid: "ae61b348556e44df9f149df9df2b921b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae61b348556e44df9f149df9df2b921b" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34787 } } }
I20260924 23:11:36.371922 22892 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:36.371838 22894 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae61b348556e44df9f149df9df2b921b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae61b348556e44df9f149df9df2b921b" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34787 } } }
I20260924 23:11:36.372092 22894 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:36.372433 22900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:11:36.372938 22900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:11:36.374421 22900 catalog_manager.cc:1347] Generated new cluster ID: 9bb2c1d71ce84d01b273a99b582d5043
I20260924 23:11:36.374472 22900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:11:36.385715 22900 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:11:36.385881 22900 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:11:36.390496 22900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae61b348556e44df9f149df9df2b921b: Generated new TSK 0
I20260924 23:11:36.390730 22900 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:11:36.396461 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34787
--builtin_ntp_servers=127.12.212.20:41835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 23:11:36.489190 22911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:36.489503 22911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:36.489542 22911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:36.491614 22911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:36.491677 22911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:36.491722 22911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:11:36.493959 22911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:41835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:36.494262 22911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:36.494640 22911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:36.500082 22919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:36.500135 22911 server_base.cc:1034] running on GCE node
W20260924 23:11:36.500106 22917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:36.500082 22916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:36.500560 22911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:36.500797 22911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:36.501955 22911 hybrid_clock.cc:648] HybridClock initialized: now 1790291496501940 us; error 33 us; skew 500 ppm
I20260924 23:11:36.503278 22911 webserver.cc:460] Webserver started at http://127.12.212.1:39961/ using document root <none> and password file <none>
I20260924 23:11:36.503504 22911 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:36.503551 22911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:36.503682 22911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:36.504750 22911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "4cbc0040908448fd98bcc40a998a9274"
format_stamp: "Formatted at 2026-09-24 23:11:36 on dist-test-slave-m0z6"
server_key: "69541f16054ac9f0ff6acea811bafdb6"
server_key_iv: "e0df2422324bbd9859a7badab63f67b1"
server_key_version: "encryptionkey@0"
I20260924 23:11:36.505086 22911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "4cbc0040908448fd98bcc40a998a9274"
format_stamp: "Formatted at 2026-09-24 23:11:36 on dist-test-slave-m0z6"
server_key: "69541f16054ac9f0ff6acea811bafdb6"
server_key_iv: "e0df2422324bbd9859a7badab63f67b1"
server_key_version: "encryptionkey@0"
I20260924 23:11:36.506372 22911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:11:36.507102 22925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:36.507279 22911 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:36.507372 22911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "4cbc0040908448fd98bcc40a998a9274"
format_stamp: "Formatted at 2026-09-24 23:11:36 on dist-test-slave-m0z6"
server_key: "69541f16054ac9f0ff6acea811bafdb6"
server_key_iv: "e0df2422324bbd9859a7badab63f67b1"
server_key_version: "encryptionkey@0"
I20260924 23:11:36.507439 22911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:36.511807 22911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:36.512053 22911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:36.512146 22911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:36.512315 22911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:36.512573 22911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:36.512609 22911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:36.512637 22911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:36.512653 22911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:36.518034 22911 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:37715
I20260924 23:11:36.518090 23038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:37715 every 8 connection(s)
I20260924 23:11:36.518404 22911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:11:36.521656 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:36.519274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:39559 (local address 127.12.212.62:34787)
0924 23:11:36.519354 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:36.519356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:36.519558 (+ 202us) server_negotiation.cc:407] Connection header received
0924 23:11:36.520034 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:36.520035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:36.520042 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:36.520108 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:36.521420 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:36.521422 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:36.521422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:36.521426 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:36.521452 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:36.521457 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:36.521520 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:36.521584 (+ 64us) server_negotiation.cc:299] Negotiation successful
0924 23:11:36.521597 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 23:11:36.521867 23041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:36.519251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:34787 (local address 127.12.212.1:39559)
0924 23:11:36.519435 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:36.519476 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:36.519926 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:36.520169 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:36.520172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:36.520181 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:36.521243 (+ 1062us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:11:36.521243 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:36.521273 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:11:36.521304 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:36.521523 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:36.521526 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:36.521528 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 23:11:36.521585 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 23:11:36.521638 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260924 23:11:36.521919 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 22911
I20260924 23:11:36.522006 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:11:36.522238 13136 external_mini_cluster.cc:1428] Setting key 437e353c2f60e3dad540e4823b90d79c
I20260924 23:11:36.522363 23039 heartbeater.cc:347] Connected to a master server at 127.12.212.62:34787
I20260924 23:11:36.522500 23039 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:36.522650 23039 heartbeater.cc:511] Master 127.12.212.62:34787 requested a full tablet report, sending...
I20260924 23:11:36.523168 22851 ts_manager.cc:194] Registered new tserver with Master: 4cbc0040908448fd98bcc40a998a9274 (127.12.212.1:37715)
I20260924 23:11:36.523713 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34787
--builtin_ntp_servers=127.12.212.20:41835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 23:11:36.603883 23042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:36.604103 23042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:36.604127 23042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:36.605295 23042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:36.605337 23042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:36.605363 23042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:11:36.606778 23042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:41835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:36.606978 23042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:36.607226 23042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:36.612232 23050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:36.612234 23048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:36.612236 23047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:36.613070 23042 server_base.cc:1034] running on GCE node
I20260924 23:11:36.613289 23042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:36.613593 23042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:36.614763 23042 hybrid_clock.cc:648] HybridClock initialized: now 1790291496614745 us; error 38 us; skew 500 ppm
I20260924 23:11:36.616014 23042 webserver.cc:460] Webserver started at http://127.12.212.2:41847/ using document root <none> and password file <none>
I20260924 23:11:36.616227 23042 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:36.616276 23042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:36.616463 23042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:36.617528 23042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "15d5598d2336431480741c58aa4dec84"
format_stamp: "Formatted at 2026-09-24 23:11:36 on dist-test-slave-m0z6"
server_key: "3f139501056ec6e5dd797c5383aa7ffe"
server_key_iv: "e62bc5ba51549d67f950858c93cd6116"
server_key_version: "encryptionkey@0"
I20260924 23:11:36.617894 23042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "15d5598d2336431480741c58aa4dec84"
format_stamp: "Formatted at 2026-09-24 23:11:36 on dist-test-slave-m0z6"
server_key: "3f139501056ec6e5dd797c5383aa7ffe"
server_key_iv: "e62bc5ba51549d67f950858c93cd6116"
server_key_version: "encryptionkey@0"
I20260924 23:11:36.619207 23042 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:11:36.619969 23056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:36.620151 23042 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 23:11:36.620234 23042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "15d5598d2336431480741c58aa4dec84"
format_stamp: "Formatted at 2026-09-24 23:11:36 on dist-test-slave-m0z6"
server_key: "3f139501056ec6e5dd797c5383aa7ffe"
server_key_iv: "e62bc5ba51549d67f950858c93cd6116"
server_key_version: "encryptionkey@0"
I20260924 23:11:36.620296 23042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:36.623946 23042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:36.624208 23042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:36.624341 23042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:36.624540 23042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:36.624830 23042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:36.624861 23042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:36.624889 23042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:36.624913 23042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:36.630491 23042 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:34011
I20260924 23:11:36.630553 23169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:34011 every 8 connection(s)
I20260924 23:11:36.630905 23042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:11:36.633953 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:36.631705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:35649 (local address 127.12.212.62:34787)
0924 23:11:36.631761 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:36.631764 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:36.632000 (+ 236us) server_negotiation.cc:407] Connection header received
0924 23:11:36.632495 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:36.632496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:36.632503 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:36.632556 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:36.633678 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:36.633680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:36.633680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:36.633685 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:36.633704 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:36.633709 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:36.633759 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:36.633865 (+ 106us) server_negotiation.cc:299] Negotiation successful
0924 23:11:36.633880 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 23:11:36.634112 23172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:36.631700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:34787 (local address 127.12.212.2:35649)
0924 23:11:36.631885 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:36.631919 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:36.632400 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:36.632625 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:36.632628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:36.632637 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:36.633505 (+ 868us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:11:36.633506 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:36.633537 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:11:36.633568 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:36.633764 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:36.633770 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:36.633773 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:11:36.633841 (+ 68us) client_negotiation.cc:240] Negotiation successful
0924 23:11:36.633909 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260924 23:11:36.634615 23170 heartbeater.cc:347] Connected to a master server at 127.12.212.62:34787
I20260924 23:11:36.634713 23170 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:36.634856 23170 heartbeater.cc:511] Master 127.12.212.62:34787 requested a full tablet report, sending...
I20260924 23:11:36.635176 22851 ts_manager.cc:194] Registered new tserver with Master: 15d5598d2336431480741c58aa4dec84 (127.12.212.2:34011)
I20260924 23:11:36.638252 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 23042
I20260924 23:11:36.638350 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:11:36.638509 13136 external_mini_cluster.cc:1428] Setting key 1539bf2b2f44eccff7535679a98055d4
I20260924 23:11:36.639397 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34787
--builtin_ntp_servers=127.12.212.20:41835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 23:11:36.721558 23173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:36.721894 23173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:36.721941 23173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:36.723587 23173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:36.723635 23173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:36.723675 23173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:11:36.725075 23173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:41835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:36.725338 23173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:36.725637 23173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:36.730538 23179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:36.730573 23178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:36.730573 23181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:36.730719 23173 server_base.cc:1034] running on GCE node
I20260924 23:11:36.731062 23173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:36.731323 23173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:36.732468 23173 hybrid_clock.cc:648] HybridClock initialized: now 1790291496732452 us; error 38 us; skew 500 ppm
I20260924 23:11:36.733803 23173 webserver.cc:460] Webserver started at http://127.12.212.3:44709/ using document root <none> and password file <none>
I20260924 23:11:36.734088 23173 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:36.734138 23173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:36.734261 23173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:36.735378 23173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "cf627f8cca6f41ccbee3caeae2037e9f"
format_stamp: "Formatted at 2026-09-24 23:11:36 on dist-test-slave-m0z6"
server_key: "308e750f07909f0541e6326b96e2e749"
server_key_iv: "70e918d51de40636393532f020398360"
server_key_version: "encryptionkey@0"
I20260924 23:11:36.735723 23173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "cf627f8cca6f41ccbee3caeae2037e9f"
format_stamp: "Formatted at 2026-09-24 23:11:36 on dist-test-slave-m0z6"
server_key: "308e750f07909f0541e6326b96e2e749"
server_key_iv: "70e918d51de40636393532f020398360"
server_key_version: "encryptionkey@0"
I20260924 23:11:36.737002 23173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:11:36.737695 23187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:36.737866 23173 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:36.737965 23173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "cf627f8cca6f41ccbee3caeae2037e9f"
format_stamp: "Formatted at 2026-09-24 23:11:36 on dist-test-slave-m0z6"
server_key: "308e750f07909f0541e6326b96e2e749"
server_key_iv: "70e918d51de40636393532f020398360"
server_key_version: "encryptionkey@0"
I20260924 23:11:36.738029 23173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:36.741520 23173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:36.741914 23173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:36.742050 23173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:36.742280 23173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:36.742595 23173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:36.742630 23173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:36.742657 23173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:36.742682 23173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:36.748221 23173 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:35707
I20260924 23:11:36.748266 23300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:35707 every 8 connection(s)
I20260924 23:11:36.748601 23173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:11:36.751701 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:36.749424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:53431 (local address 127.12.212.62:34787)
0924 23:11:36.749504 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:36.749506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:36.749649 (+ 143us) server_negotiation.cc:407] Connection header received
0924 23:11:36.750164 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:36.750165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:36.750172 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:36.750216 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:36.751454 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:36.751455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:36.751456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:11:36.751461 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:36.751478 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:11:36.751482 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:36.751534 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:36.751611 (+ 77us) server_negotiation.cc:299] Negotiation successful
0924 23:11:36.751627 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 23:11:36.751892 23303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:36.749322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:34787 (local address 127.12.212.3:53431)
0924 23:11:36.749530 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:36.749570 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:36.750061 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:36.750325 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:36.750328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:36.750337 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:36.751291 (+ 954us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:11:36.751292 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:36.751323 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:11:36.751356 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:36.751541 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:36.751546 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:36.751549 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:11:36.751616 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 23:11:36.751682 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260924 23:11:36.752473 23301 heartbeater.cc:347] Connected to a master server at 127.12.212.62:34787
I20260924 23:11:36.752554 23301 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:36.752696 23301 heartbeater.cc:511] Master 127.12.212.62:34787 requested a full tablet report, sending...
I20260924 23:11:36.753042 22851 ts_manager.cc:194] Registered new tserver with Master: cf627f8cca6f41ccbee3caeae2037e9f (127.12.212.3:35707)
I20260924 23:11:36.754060 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 23173
I20260924 23:11:36.754134 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:11:36.754281 13136 external_mini_cluster.cc:1428] Setting key 1aa45f252dbab52f6bcc1841bcc8cd63
I20260924 23:11:36.755141 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 23:11:36.756808 22891 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:11:36.756384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39528 (local address 127.12.212.62:34787)
0924 23:11:36.756451 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:36.756453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:36.756545 (+ 92us) server_negotiation.cc:407] Connection header received
0924 23:11:36.756591 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:36.756592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:36.756596 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:36.756637 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:36.756742 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39528: BlockingRecv error: recv got EOF from 127.0.0.1:39528 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 23:11:36.756867 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 22911
I20260924 23:11:36.761283 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 23042
I20260924 23:11:36.764950 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 23173
I20260924 23:11:36.768662 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 22820
2026-09-24T23:11:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (539 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:11:36 dist-test-slave-m0z6 krb5kdc[23311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:11:36 dist-test-slave-m0z6 krb5kdc[23311](info): set up 2 sockets
Sep 24 23:11:36 dist-test-slave-m0z6 krb5kdc[23311](info): commencing operation
krb5kdc: starting...
W20260924 23:11:37.719367 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:11:37 dist-test-slave-m0z6 krb5kdc[23311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:11:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:11:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:11:37.837634 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:34761
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:46847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:34761
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:37.919121 23324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:37.919324 23324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:37.919346 23324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:37.920473 23324 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:11:37.920513 23324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:37.920526 23324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:37.920538 23324 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:11:37.920548 23324 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:11:37.921967 23324 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:46847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:34761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:34761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:37.922174 23324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:37.922349 23324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:11:37.927192 23324 server_base.cc:1034] running on GCE node
W20260924 23:11:37.927163 23330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:37.927218 23329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:37.927163 23332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:37.927654 23324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:37.927949 23324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:37.929102 23324 hybrid_clock.cc:648] HybridClock initialized: now 1790291497929084 us; error 30 us; skew 500 ppm
Sep 24 23:11:37 dist-test-slave-m0z6 krb5kdc[23311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291497, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:37.931694 23324 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:11:37.932163 23324 webserver.cc:460] Webserver started at http://127.12.212.62:32779/ using document root <none> and password file <none>
I20260924 23:11:37.932389 23324 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:37.932449 23324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:37.932574 23324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:37.933665 23324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "b610eee9ba72400caad499003ca9ed0a"
format_stamp: "Formatted at 2026-09-24 23:11:37 on dist-test-slave-m0z6"
server_key: "9f7682daa461e3bb767d9da491d23061"
server_key_iv: "5141aa3f24e7ea41f439c00275d6a47d"
server_key_version: "encryptionkey@0"
I20260924 23:11:37.934018 23324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "b610eee9ba72400caad499003ca9ed0a"
format_stamp: "Formatted at 2026-09-24 23:11:37 on dist-test-slave-m0z6"
server_key: "9f7682daa461e3bb767d9da491d23061"
server_key_iv: "5141aa3f24e7ea41f439c00275d6a47d"
server_key_version: "encryptionkey@0"
I20260924 23:11:37.935527 23324 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:11:37.936363 23339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:37.936607 23324 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:37.936704 23324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "b610eee9ba72400caad499003ca9ed0a"
format_stamp: "Formatted at 2026-09-24 23:11:37 on dist-test-slave-m0z6"
server_key: "9f7682daa461e3bb767d9da491d23061"
server_key_iv: "5141aa3f24e7ea41f439c00275d6a47d"
server_key_version: "encryptionkey@0"
I20260924 23:11:37.936772 23324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:37.948037 23324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:37.950428 23324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:37.950642 23324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:37.954473 23324 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:34761
I20260924 23:11:37.954517 23391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:34761 every 8 connection(s)
I20260924 23:11:37.954907 23324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:11:37.955953 23392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:37.958302 23392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a: Bootstrap starting.
I20260924 23:11:37.958956 23392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:37.959232 23392 log.cc:824] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:37.960124 23392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a: No bootstrap required, opened a new log
I20260924 23:11:37.961884 23392 raft_consensus.cc:348] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b610eee9ba72400caad499003ca9ed0a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34761 } }
I20260924 23:11:37.962064 23392 raft_consensus.cc:374] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:37.962090 23392 raft_consensus.cc:729] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b610eee9ba72400caad499003ca9ed0a, State: Initialized, Role: FOLLOWER
I20260924 23:11:37.962237 23392 consensus_queue.cc:260] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b610eee9ba72400caad499003ca9ed0a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34761 } }
I20260924 23:11:37.962317 23392 raft_consensus.cc:388] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:11:37.962343 23392 raft_consensus.cc:482] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:11:37.962384 23392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:37.962584 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 23324
I20260924 23:11:37.962673 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:11:37.962869 13136 external_mini_cluster.cc:1428] Setting key b55ca8f08e4bc9915c57b78ebbf81a4b
I20260924 23:11:37.963397 23392 raft_consensus.cc:504] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b610eee9ba72400caad499003ca9ed0a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34761 } }
I20260924 23:11:37.963518 23392 leader_election.cc:302] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b610eee9ba72400caad499003ca9ed0a; no voters:
I20260924 23:11:37.963671 23392 leader_election.cc:288] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:11:37.963701 23397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:37.963860 23397 raft_consensus.cc:686] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a [term 1 LEADER]: Becoming Leader. State: Replica: b610eee9ba72400caad499003ca9ed0a, State: Running, Role: LEADER
I20260924 23:11:37.963919 23392 sys_catalog.cc:564] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:11:37.963971 23397 consensus_queue.cc:237] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b610eee9ba72400caad499003ca9ed0a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34761 } }
I20260924 23:11:37.964406 23398 sys_catalog.cc:455] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b610eee9ba72400caad499003ca9ed0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b610eee9ba72400caad499003ca9ed0a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34761 } } }
I20260924 23:11:37.964480 23398 sys_catalog.cc:458] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:37.964712 23399 sys_catalog.cc:455] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b610eee9ba72400caad499003ca9ed0a. Latest consensus state: current_term: 1 leader_uuid: "b610eee9ba72400caad499003ca9ed0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b610eee9ba72400caad499003ca9ed0a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34761 } } }
I20260924 23:11:37.965081 23399 sys_catalog.cc:458] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a [sys.catalog]: This master's current role is: LEADER
I20260924 23:11:37.965124 23403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:11:37.965682 23403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:11:37.967610 23403 catalog_manager.cc:1347] Generated new cluster ID: 828059f03e3a40f2ab8a678876e4b359
I20260924 23:11:37.967679 23403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 23:11:37 dist-test-slave-m0z6 krb5kdc[23311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:37.975101 23403 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:11:37.975841 23403 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:11:37.976349 23396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:37.963357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49090 (local address 127.12.212.62:34761)
0924 23:11:37.963589 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:37.963599 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:37.963638 (+ 39us) server_negotiation.cc:407] Connection header received
0924 23:11:37.963777 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:37.963784 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:37.963845 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:37.964789 (+ 944us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:37.964896 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:37.966358 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:37.967418 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:37.968039 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:37.968154 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:37.972135 (+ 3981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:37.972141 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:37.972141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:37.972159 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:37.974244 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:37.974964 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:37.974968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:37.974969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:37.975106 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:37.975387 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:37.975388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:37.975389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:37.975644 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:37.975807 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:37.975917 (+ 110us) server_negotiation.cc:299] Negotiation successful
0924 23:11:37.976108 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":100,"threads_started":1}
I20260924 23:11:37.984457 23403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a: Generated new TSK 0
I20260924 23:11:37.984720 23403 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
I20260924 23:11:38.019878 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34761
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:46847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:38.122570 23418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:38.122824 23418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:38.122853 23418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:38.124163 23418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:38.124207 23418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:38.124234 23418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:11:38.125675 23418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:46847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:38.125895 23418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:38.126111 23418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:38.130849 23424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:38.130921 23418 server_base.cc:1034] running on GCE node
W20260924 23:11:38.130846 23423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:38.130846 23426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:38.131237 23418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:38.131479 23418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:38.132647 23418 hybrid_clock.cc:648] HybridClock initialized: now 1790291498132632 us; error 25 us; skew 500 ppm
Sep 24 23:11:38 dist-test-slave-m0z6 krb5kdc[23311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291498, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:38.135042 23418 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:11:38.135565 23418 webserver.cc:460] Webserver started at http://127.12.212.1:44885/ using document root <none> and password file <none>
I20260924 23:11:38.135769 23418 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:38.135818 23418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:38.135942 23418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:38.137068 23418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "4d4362283c7b4a8e82cff9984cfc63c8"
format_stamp: "Formatted at 2026-09-24 23:11:38 on dist-test-slave-m0z6"
server_key: "441820c31accf08eacd0693323c7a46d"
server_key_iv: "7bc040ee031dbb5d0c4a6d3af23ec156"
server_key_version: "encryptionkey@0"
I20260924 23:11:38.137437 23418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "4d4362283c7b4a8e82cff9984cfc63c8"
format_stamp: "Formatted at 2026-09-24 23:11:38 on dist-test-slave-m0z6"
server_key: "441820c31accf08eacd0693323c7a46d"
server_key_iv: "7bc040ee031dbb5d0c4a6d3af23ec156"
server_key_version: "encryptionkey@0"
I20260924 23:11:38.138788 23418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:11:38.139569 23433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:38.139775 23418 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:11:38.139882 23418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "4d4362283c7b4a8e82cff9984cfc63c8"
format_stamp: "Formatted at 2026-09-24 23:11:38 on dist-test-slave-m0z6"
server_key: "441820c31accf08eacd0693323c7a46d"
server_key_iv: "7bc040ee031dbb5d0c4a6d3af23ec156"
server_key_version: "encryptionkey@0"
I20260924 23:11:38.139962 23418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:38.159482 23418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:38.161543 23418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:38.161691 23418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:38.161939 23418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:38.162231 23418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:38.162264 23418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:38.162312 23418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:38.162334 23418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:38.168037 23418 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:43293
I20260924 23:11:38.168078 23546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:43293 every 8 connection(s)
I20260924 23:11:38.168397 23418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
Sep 24 23:11:38 dist-test-slave-m0z6 krb5kdc[23311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291498, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:38.175864 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 23418
I20260924 23:11:38.176014 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:11:38.176205 13136 external_mini_cluster.cc:1428] Setting key 6e320ae930e6daa486fa431909ed8e47
I20260924 23:11:38.178800 23396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:38.169118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:51829 (local address 127.12.212.62:34761)
0924 23:11:38.169172 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:38.169174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:38.169369 (+ 195us) server_negotiation.cc:407] Connection header received
0924 23:11:38.169911 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:38.169912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:38.169922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:38.169988 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:38.171034 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.171565 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.172605 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.172813 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.172878 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:38.175449 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:38.175450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:38.175451 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:38.175460 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:38.177281 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:38.177746 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:38.177748 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:38.177748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:38.177808 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:38.178067 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:38.178068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:38.178068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:38.178191 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:38.178250 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:38.178563 (+ 313us) server_negotiation.cc:299] Negotiation successful
0924 23:11:38.178673 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 23:11:38.178992 23549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:38.169087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:34761 (local address 127.12.212.1:51829)
0924 23:11:38.169249 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:38.169292 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:38.169771 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:38.170058 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:38.170062 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:38.170630 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:38.170929 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:38.170934 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.171668 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.171670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:38.172506 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:38.172509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.172938 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.172939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:38.173083 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:38.173640 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:38.173649 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:38.175328 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:38.177372 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:38.177393 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:38.177397 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:38.177670 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:38.177869 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:38.177870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:38.177870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:38.178006 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:38.178251 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:38.178260 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:38.178432 (+ 172us) client_negotiation.cc:769] Sending connection context
0924 23:11:38.178530 (+ 98us) client_negotiation.cc:240] Negotiation successful
0924 23:11:38.178753 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
I20260924 23:11:38.179594 23547 heartbeater.cc:347] Connected to a master server at 127.12.212.62:34761
I20260924 23:11:38.179698 23547 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:38.179947 23547 heartbeater.cc:511] Master 127.12.212.62:34761 requested a full tablet report, sending...
I20260924 23:11:38.180648 23356 ts_manager.cc:194] Registered new tserver with Master: 4d4362283c7b4a8e82cff9984cfc63c8 (127.12.212.1:43293)
I20260924 23:11:38.181298 23356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:51829
WARNING: no policy specified for kudu/127.12.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
I20260924 23:11:38.202557 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34761
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:46847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:38.284315 23552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:38.284528 23552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:38.284550 23552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:38.285672 23552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:38.285713 23552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:38.285737 23552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:11:38.287271 23552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:46847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:38.287497 23552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:38.287714 23552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:38.292768 23557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:38.292775 23560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:38.292766 23558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:38.293020 23552 server_base.cc:1034] running on GCE node
I20260924 23:11:38.293196 23552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:38.293411 23552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:38.294588 23552 hybrid_clock.cc:648] HybridClock initialized: now 1790291498294567 us; error 30 us; skew 500 ppm
Sep 24 23:11:38 dist-test-slave-m0z6 krb5kdc[23311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291498, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:38.297214 23552 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:11:38.297748 23552 webserver.cc:460] Webserver started at http://127.12.212.2:42065/ using document root <none> and password file <none>
I20260924 23:11:38.297991 23552 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:38.298043 23552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:38.298169 23552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:38.299274 23552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "977a59caf42e4be4866756fba22de467"
format_stamp: "Formatted at 2026-09-24 23:11:38 on dist-test-slave-m0z6"
server_key: "43d20a51f0c965910227a7b2fbb2af9d"
server_key_iv: "e21de975e90da1a015ac3261aac63469"
server_key_version: "encryptionkey@0"
I20260924 23:11:38.299614 23552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "977a59caf42e4be4866756fba22de467"
format_stamp: "Formatted at 2026-09-24 23:11:38 on dist-test-slave-m0z6"
server_key: "43d20a51f0c965910227a7b2fbb2af9d"
server_key_iv: "e21de975e90da1a015ac3261aac63469"
server_key_version: "encryptionkey@0"
I20260924 23:11:38.300943 23552 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:11:38.301677 23567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:38.301853 23552 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:38.301961 23552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "977a59caf42e4be4866756fba22de467"
format_stamp: "Formatted at 2026-09-24 23:11:38 on dist-test-slave-m0z6"
server_key: "43d20a51f0c965910227a7b2fbb2af9d"
server_key_iv: "e21de975e90da1a015ac3261aac63469"
server_key_version: "encryptionkey@0"
I20260924 23:11:38.302028 23552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:38.318384 23552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:38.320564 23552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:38.320710 23552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:38.320941 23552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:38.321259 23552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:38.321293 23552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:38.321333 23552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:38.321357 23552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:38.327180 23552 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:34531
I20260924 23:11:38.327224 23680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:34531 every 8 connection(s)
I20260924 23:11:38.327566 23552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:11:38.328200 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 23552
I20260924 23:11:38.328289 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:11:38.328505 13136 external_mini_cluster.cc:1428] Setting key 69f8207bdae34fbb280d8d98d19885b7
Sep 24 23:11:38 dist-test-slave-m0z6 krb5kdc[23311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291498, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.212.3@KRBTEST.COM; defaulting to no policy
I20260924 23:11:38.338183 23396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:38.328303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:46025 (local address 127.12.212.62:34761)
0924 23:11:38.328375 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:38.328377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:38.328626 (+ 249us) server_negotiation.cc:407] Connection header received
0924 23:11:38.329220 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:38.329220 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:38.329230 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:38.329258 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:38.330528 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.331262 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.332549 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.332766 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.332811 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:38.335254 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:38.335255 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:38.335256 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:38.335263 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:38.336666 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:38.337141 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:38.337142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:38.337143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:38.337208 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:38.337533 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:38.337534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:38.337534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:38.337660 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:38.337723 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:38.338008 (+ 285us) server_negotiation.cc:299] Negotiation successful
0924 23:11:38.338090 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 23:11:38.338338 23683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:38.328272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:34761 (local address 127.12.212.2:46025)
0924 23:11:38.328435 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:38.328490 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:38.329127 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:38.329344 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:38.329349 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:38.330086 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:38.330370 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:38.330377 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.331356 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.331357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:38.332445 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:38.332448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.332835 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.332836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:38.332981 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:38.333480 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:38.333492 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:38.335153 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:38.336750 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:38.336758 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:38.336761 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:38.337056 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:38.337259 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:38.337260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:38.337260 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:38.337404 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:38.337732 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:38.337739 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:38.337878 (+ 139us) client_negotiation.cc:769] Sending connection context
0924 23:11:38.337970 (+ 92us) client_negotiation.cc:240] Negotiation successful
0924 23:11:38.338112 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260924 23:11:38.338841 23681 heartbeater.cc:347] Connected to a master server at 127.12.212.62:34761
I20260924 23:11:38.338938 23681 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:38.339156 23681 heartbeater.cc:511] Master 127.12.212.62:34761 requested a full tablet report, sending...
I20260924 23:11:38.339547 23356 ts_manager.cc:194] Registered new tserver with Master: 977a59caf42e4be4866756fba22de467 (127.12.212.2:34531)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.3@KRBTEST.COM" created.
I20260924 23:11:38.340070 23356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:46025
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
I20260924 23:11:38.353884 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34761
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:46847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:11:38.434212 23686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:11:38.434453 23686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:11:38.434487 23686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:11:38.435798 23686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:11:38.435863 23686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:11:38.435904 23686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:11:38.437573 23686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:46847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:34761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:11:38.437841 23686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:11:38.438122 23686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:11:38.443145 23694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:38.443151 23692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:11:38.443161 23691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:11:38.443543 23686 server_base.cc:1034] running on GCE node
I20260924 23:11:38.443753 23686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:11:38.443979 23686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:11:38.445118 23686 hybrid_clock.cc:648] HybridClock initialized: now 1790291498445104 us; error 26 us; skew 500 ppm
Sep 24 23:11:38 dist-test-slave-m0z6 krb5kdc[23311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291498, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:11:38.447521 23686 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:11:38.447942 23686 webserver.cc:460] Webserver started at http://127.12.212.3:35865/ using document root <none> and password file <none>
I20260924 23:11:38.448159 23686 fs_manager.cc:359] Metadata directory not provided
I20260924 23:11:38.448218 23686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:11:38.448325 23686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:11:38.449359 23686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "72623fccae984f90b9dacc70042ff743"
format_stamp: "Formatted at 2026-09-24 23:11:38 on dist-test-slave-m0z6"
server_key: "09ce2199386462bb4e0d522dccda3a64"
server_key_iv: "053bf4ab6bbcf603e7feeba5441aa35a"
server_key_version: "encryptionkey@0"
I20260924 23:11:38.449680 23686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "72623fccae984f90b9dacc70042ff743"
format_stamp: "Formatted at 2026-09-24 23:11:38 on dist-test-slave-m0z6"
server_key: "09ce2199386462bb4e0d522dccda3a64"
server_key_iv: "053bf4ab6bbcf603e7feeba5441aa35a"
server_key_version: "encryptionkey@0"
I20260924 23:11:38.450914 23686 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:11:38.451599 23701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:38.451737 23686 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:11:38.451810 23686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "72623fccae984f90b9dacc70042ff743"
format_stamp: "Formatted at 2026-09-24 23:11:38 on dist-test-slave-m0z6"
server_key: "09ce2199386462bb4e0d522dccda3a64"
server_key_iv: "053bf4ab6bbcf603e7feeba5441aa35a"
server_key_version: "encryptionkey@0"
I20260924 23:11:38.451872 23686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:11:38.475062 23686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:11:38.477506 23686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:11:38.477655 23686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:11:38.477893 23686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:11:38.478237 23686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:11:38.478273 23686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:38.478302 23686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:11:38.478328 23686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:11:38.484177 23686 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:36643
I20260924 23:11:38.484251 23814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:36643 every 8 connection(s)
I20260924 23:11:38.484642 23686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:11:38.489079 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 23686
I20260924 23:11:38.489183 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:11:38.489364 13136 external_mini_cluster.cc:1428] Setting key 23e40bb3124e489164277807e6f0104e
Sep 24 23:11:38 dist-test-slave-m0z6 krb5kdc[23311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291498, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:11:38.495577 23396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:38.485415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:55919 (local address 127.12.212.62:34761)
0924 23:11:38.485457 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:38.485459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:38.485750 (+ 291us) server_negotiation.cc:407] Connection header received
0924 23:11:38.486249 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:38.486251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:38.486262 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:38.486332 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:38.487551 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.488131 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.489397 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.489633 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.489690 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:38.492502 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:38.492504 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:38.492504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:38.492513 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:38.493919 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:38.494418 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:38.494420 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:38.494420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:38.494517 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:38.494914 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:38.494915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:38.494915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:38.495077 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:38.495147 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:38.495416 (+ 269us) server_negotiation.cc:299] Negotiation successful
0924 23:11:38.495492 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 23:11:38.495811 23817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:38.485397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:34761 (local address 127.12.212.3:55919)
0924 23:11:38.485567 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:38.485614 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:38.486147 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:38.486457 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:38.486461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:38.487198 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:11:38.487421 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:38.487427 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.488272 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.488273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:38.489303 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:38.489305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.489720 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.489721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:38.489853 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:38.490437 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:11:38.490446 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:11:38.492341 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:11:38.494027 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:38.494037 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:38.494039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:38.494308 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:38.494618 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:11:38.494619 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:11:38.494619 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:11:38.494840 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:11:38.495148 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:11:38.495154 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:11:38.495301 (+ 147us) client_negotiation.cc:769] Sending connection context
0924 23:11:38.495384 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 23:11:38.495580 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":68,"threads_started":1}
I20260924 23:11:38.496345 23815 heartbeater.cc:347] Connected to a master server at 127.12.212.62:34761
I20260924 23:11:38.496447 23815 heartbeater.cc:464] Registering TS with master...
I20260924 23:11:38.496683 23815 heartbeater.cc:511] Master 127.12.212.62:34761 requested a full tablet report, sending...
I20260924 23:11:38.497077 23356 ts_manager.cc:194] Registered new tserver with Master: 72623fccae984f90b9dacc70042ff743 (127.12.212.3:36643)
I20260924 23:11:38.497459 23356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:55919
I20260924 23:11:38.501142 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 23:11:38.508554 23396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:38.502415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49106 (local address 127.12.212.62:34761)
0924 23:11:38.502470 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:38.502472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:38.502581 (+ 109us) server_negotiation.cc:407] Connection header received
0924 23:11:38.502652 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:38.502653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:38.502663 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:38.502744 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:11:38.503305 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.503802 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.504504 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.504695 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.504751 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:38.506071 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:11:38.506072 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:11:38.506073 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:11:38.506081 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:11:38.507395 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:38.507744 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:38.507745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:38.507745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:38.507802 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:11:38.507974 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:11:38.507975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:11:38.507975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:11:38.508135 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:11:38.508226 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:38.508371 (+ 145us) server_negotiation.cc:299] Negotiation successful
0924 23:11:38.508458 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 23:11:38.510018 23356 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49106:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:11:38.511713 23356 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:11:38.518369 23828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:38.514240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56092 (local address 127.12.212.3:36643)
0924 23:11:38.514876 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:38.514880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:38.514891 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:11:38.514943 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:38.514945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:38.514962 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:38.515042 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:38.515252 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.515966 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.517453 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.517983 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.518054 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:38.518118 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:38.518239 (+ 121us) server_negotiation.cc:299] Negotiation successful
0924 23:11:38.518287 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":72,"threads_started":1}
I20260924 23:11:38.518527 23829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:38.514358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:43293 (local address 127.0.0.1:37454)
0924 23:11:38.514490 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:38.514501 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:38.514553 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:38.515140 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:38.515141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:38.515143 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:38.515193 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:38.515195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.516459 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.516460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:38.517321 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:38.517323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.518269 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.518270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:38.518334 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:38.518339 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 23:11:38.518368 (+ 29us) client_negotiation.cc:240] Negotiation successful
0924 23:11:38.518407 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260924 23:11:38.518606 23826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:38.514143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:34531 (local address 127.0.0.1:44830)
0924 23:11:38.514308 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:38.514354 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:38.514424 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:38.514867 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:38.514870 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:38.514873 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:38.514957 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:38.514959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.516359 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.516360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:38.517438 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:38.517440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.518281 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.518282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:38.518453 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:38.518456 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:11:38.518484 (+ 28us) client_negotiation.cc:240] Negotiation successful
0924 23:11:38.518529 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":102,"threads_started":1}
I20260924 23:11:38.518657 23831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:38.514790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37454 (local address 127.12.212.1:43293)
0924 23:11:38.514964 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:38.514968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:38.514976 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:11:38.515010 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:38.515012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:38.515026 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:38.515125 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:38.515281 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.516372 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.517401 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.518165 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.518452 (+ 287us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:38.518523 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:38.518548 (+ 25us) server_negotiation.cc:299] Negotiation successful
0924 23:11:38.518586 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":77,"threads_started":1}
I20260924 23:11:38.518908 23830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:38.514513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44830 (local address 127.12.212.2:34531)
0924 23:11:38.514663 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:38.514667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:38.514676 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:11:38.514739 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:38.514741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:38.514757 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:38.515029 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:38.515101 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.516249 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.517509 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.518202 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.518742 (+ 540us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:38.518798 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:38.518820 (+ 22us) server_negotiation.cc:299] Negotiation successful
0924 23:11:38.518853 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260924 23:11:38.518373 23827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:38.514141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:36643 (local address 127.0.0.1:56092)
0924 23:11:38.514308 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:38.514354 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:38.514424 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:38.515119 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:38.515120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:38.515122 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:38.515176 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:38.515178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.516059 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.516060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:38.517357 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:38.517360 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.518061 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.518061 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:38.518138 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:38.518153 (+ 15us) client_negotiation.cc:769] Sending connection context
0924 23:11:38.518220 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 23:11:38.518276 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":23,"thread_start_us":64,"threads_started":1}
I20260924 23:11:38.519451 23481 tablet_service.cc:1467] Processing CreateTablet for tablet 0222a1a6d66640e38cb5b274c7e7d844 (DEFAULT_TABLE table=test-table [id=06703e8ba9a3462db904d1daaa2cef60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:38.519562 23615 tablet_service.cc:1467] Processing CreateTablet for tablet 0222a1a6d66640e38cb5b274c7e7d844 (DEFAULT_TABLE table=test-table [id=06703e8ba9a3462db904d1daaa2cef60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:38.519797 23481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0222a1a6d66640e38cb5b274c7e7d844. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:38.519709 23749 tablet_service.cc:1467] Processing CreateTablet for tablet 0222a1a6d66640e38cb5b274c7e7d844 (DEFAULT_TABLE table=test-table [id=06703e8ba9a3462db904d1daaa2cef60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:11:38.519902 23615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0222a1a6d66640e38cb5b274c7e7d844. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:38.520006 23749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0222a1a6d66640e38cb5b274c7e7d844. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:11:38.522312 23832 tablet_bootstrap.cc:492] T 0222a1a6d66640e38cb5b274c7e7d844 P 4d4362283c7b4a8e82cff9984cfc63c8: Bootstrap starting.
I20260924 23:11:38.523021 23832 tablet_bootstrap.cc:654] T 0222a1a6d66640e38cb5b274c7e7d844 P 4d4362283c7b4a8e82cff9984cfc63c8: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:38.523104 23833 tablet_bootstrap.cc:492] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743: Bootstrap starting.
I20260924 23:11:38.523355 23834 tablet_bootstrap.cc:492] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467: Bootstrap starting.
I20260924 23:11:38.523774 23833 tablet_bootstrap.cc:654] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:38.523890 23832 log.cc:824] T 0222a1a6d66640e38cb5b274c7e7d844 P 4d4362283c7b4a8e82cff9984cfc63c8: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:38.523938 23834 tablet_bootstrap.cc:654] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467: Neither blocks nor log segments found. Creating new log.
I20260924 23:11:38.524374 23833 log.cc:824] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:38.524415 23834 log.cc:824] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467: Log is configured to *not* fsync() on all Append() calls
I20260924 23:11:38.524685 23832 tablet_bootstrap.cc:492] T 0222a1a6d66640e38cb5b274c7e7d844 P 4d4362283c7b4a8e82cff9984cfc63c8: No bootstrap required, opened a new log
I20260924 23:11:38.524758 23832 ts_tablet_manager.cc:1397] T 0222a1a6d66640e38cb5b274c7e7d844 P 4d4362283c7b4a8e82cff9984cfc63c8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260924 23:11:38.525017 23833 tablet_bootstrap.cc:492] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743: No bootstrap required, opened a new log
I20260924 23:11:38.525074 23833 ts_tablet_manager.cc:1397] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:11:38.525104 23834 tablet_bootstrap.cc:492] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467: No bootstrap required, opened a new log
I20260924 23:11:38.525169 23834 ts_tablet_manager.cc:1397] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:11:38.526571 23833 raft_consensus.cc:348] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72623fccae984f90b9dacc70042ff743" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36643 } } peers { permanent_uuid: "977a59caf42e4be4866756fba22de467" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34531 } } peers { permanent_uuid: "4d4362283c7b4a8e82cff9984cfc63c8" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 43293 } }
I20260924 23:11:38.526742 23833 raft_consensus.cc:374] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:38.526798 23833 raft_consensus.cc:729] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72623fccae984f90b9dacc70042ff743, State: Initialized, Role: FOLLOWER
I20260924 23:11:38.526726 23832 raft_consensus.cc:348] T 0222a1a6d66640e38cb5b274c7e7d844 P 4d4362283c7b4a8e82cff9984cfc63c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72623fccae984f90b9dacc70042ff743" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36643 } } peers { permanent_uuid: "977a59caf42e4be4866756fba22de467" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34531 } } peers { permanent_uuid: "4d4362283c7b4a8e82cff9984cfc63c8" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 43293 } }
I20260924 23:11:38.526906 23832 raft_consensus.cc:374] T 0222a1a6d66640e38cb5b274c7e7d844 P 4d4362283c7b4a8e82cff9984cfc63c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:38.526945 23832 raft_consensus.cc:729] T 0222a1a6d66640e38cb5b274c7e7d844 P 4d4362283c7b4a8e82cff9984cfc63c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4362283c7b4a8e82cff9984cfc63c8, State: Initialized, Role: FOLLOWER
I20260924 23:11:38.526912 23833 consensus_queue.cc:260] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72623fccae984f90b9dacc70042ff743" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36643 } } peers { permanent_uuid: "977a59caf42e4be4866756fba22de467" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34531 } } peers { permanent_uuid: "4d4362283c7b4a8e82cff9984cfc63c8" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 43293 } }
I20260924 23:11:38.527083 23832 consensus_queue.cc:260] T 0222a1a6d66640e38cb5b274c7e7d844 P 4d4362283c7b4a8e82cff9984cfc63c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72623fccae984f90b9dacc70042ff743" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36643 } } peers { permanent_uuid: "977a59caf42e4be4866756fba22de467" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34531 } } peers { permanent_uuid: "4d4362283c7b4a8e82cff9984cfc63c8" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 43293 } }
I20260924 23:11:38.527195 23833 ts_tablet_manager.cc:1428] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:11:38.527120 23834 raft_consensus.cc:348] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72623fccae984f90b9dacc70042ff743" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36643 } } peers { permanent_uuid: "977a59caf42e4be4866756fba22de467" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34531 } } peers { permanent_uuid: "4d4362283c7b4a8e82cff9984cfc63c8" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 43293 } }
I20260924 23:11:38.527240 23815 heartbeater.cc:503] Master 127.12.212.62:34761 was elected leader, sending a full tablet report...
I20260924 23:11:38.527249 23834 raft_consensus.cc:374] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:11:38.527281 23834 raft_consensus.cc:729] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977a59caf42e4be4866756fba22de467, State: Initialized, Role: FOLLOWER
I20260924 23:11:38.527366 23832 ts_tablet_manager.cc:1428] T 0222a1a6d66640e38cb5b274c7e7d844 P 4d4362283c7b4a8e82cff9984cfc63c8: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260924 23:11:38.527385 23834 consensus_queue.cc:260] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72623fccae984f90b9dacc70042ff743" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36643 } } peers { permanent_uuid: "977a59caf42e4be4866756fba22de467" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34531 } } peers { permanent_uuid: "4d4362283c7b4a8e82cff9984cfc63c8" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 43293 } }
I20260924 23:11:38.527482 23547 heartbeater.cc:503] Master 127.12.212.62:34761 was elected leader, sending a full tablet report...
I20260924 23:11:38.527637 23834 ts_tablet_manager.cc:1428] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260924 23:11:38.527688 23681 heartbeater.cc:503] Master 127.12.212.62:34761 was elected leader, sending a full tablet report...
W20260924 23:11:38.578161 23682 tablet.cc:2369] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 23:11:38.669281 23548 tablet.cc:2369] T 0222a1a6d66640e38cb5b274c7e7d844 P 4d4362283c7b4a8e82cff9984cfc63c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:11:38.669272 23840 raft_consensus.cc:482] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:11:38.669420 23840 raft_consensus.cc:504] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72623fccae984f90b9dacc70042ff743" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36643 } } peers { permanent_uuid: "977a59caf42e4be4866756fba22de467" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34531 } } peers { permanent_uuid: "4d4362283c7b4a8e82cff9984cfc63c8" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 43293 } }
I20260924 23:11:38.669795 23840 leader_election.cc:288] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72623fccae984f90b9dacc70042ff743 (127.12.212.3:36643), 4d4362283c7b4a8e82cff9984cfc63c8 (127.12.212.1:43293)
I20260924 23:11:38.673985 23683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:38.669996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:36643 (local address 127.12.212.2:56323)
0924 23:11:38.670063 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:38.670076 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:38.670122 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:38.670286 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:38.670288 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:38.670289 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:38.670363 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:38.670365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.671473 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.671475 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:38.672782 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:38.672786 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.673670 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.673671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:38.673769 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:38.673772 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:11:38.673829 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 23:11:38.673880 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":24}
I20260924 23:11:38.674018 23828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:38.670104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:56323 (local address 127.12.212.3:36643)
0924 23:11:38.670152 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:38.670154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:38.670161 (+ 7us) server_negotiation.cc:407] Connection header received
0924 23:11:38.670187 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:38.670187 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:38.670194 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:38.670258 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:38.670533 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.671302 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.672928 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.673553 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.673626 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:38.673635 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:38.673884 (+ 249us) server_negotiation.cc:299] Negotiation successful
0924 23:11:38.673944 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 23:11:38.674021 23831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:38.670072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:50635 (local address 127.12.212.1:43293)
0924 23:11:38.670115 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:38.670117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:38.670295 (+ 178us) server_negotiation.cc:407] Connection header received
0924 23:11:38.670325 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:38.670326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:38.670334 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:38.670379 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:11:38.670529 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.671302 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.672947 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.673528 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.673625 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:38.673635 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:38.673883 (+ 248us) server_negotiation.cc:299] Negotiation successful
0924 23:11:38.673940 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 23:11:38.673987 23841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:38.669996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:43293 (local address 127.12.212.2:50635)
0924 23:11:38.670222 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:38.670230 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:11:38.670285 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:11:38.670405 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:11:38.670406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:11:38.670407 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:11:38.670465 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:38.670466 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.671416 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.671418 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:38.672782 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:11:38.672786 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.673670 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.673671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:11:38.673761 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:38.673766 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 23:11:38.673829 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 23:11:38.673880 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":98,"thread_start_us":136,"threads_started":1}
I20260924 23:11:38.674403 23769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0222a1a6d66640e38cb5b274c7e7d844" candidate_uuid: "977a59caf42e4be4866756fba22de467" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72623fccae984f90b9dacc70042ff743" is_pre_election: true
I20260924 23:11:38.674757 23769 raft_consensus.cc:2454] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 977a59caf42e4be4866756fba22de467 in term 0.
I20260924 23:11:38.674980 23501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0222a1a6d66640e38cb5b274c7e7d844" candidate_uuid: "977a59caf42e4be4866756fba22de467" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4362283c7b4a8e82cff9984cfc63c8" is_pre_election: true
I20260924 23:11:38.675019 23571 leader_election.cc:302] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72623fccae984f90b9dacc70042ff743, 977a59caf42e4be4866756fba22de467; no voters:
I20260924 23:11:38.675122 23501 raft_consensus.cc:2454] T 0222a1a6d66640e38cb5b274c7e7d844 P 4d4362283c7b4a8e82cff9984cfc63c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 977a59caf42e4be4866756fba22de467 in term 0.
I20260924 23:11:38.675175 23840 raft_consensus.cc:2781] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:11:38.675225 23840 raft_consensus.cc:482] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:11:38.675246 23840 raft_consensus.cc:3037] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:38.675959 23840 raft_consensus.cc:504] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72623fccae984f90b9dacc70042ff743" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36643 } } peers { permanent_uuid: "977a59caf42e4be4866756fba22de467" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34531 } } peers { permanent_uuid: "4d4362283c7b4a8e82cff9984cfc63c8" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 43293 } }
I20260924 23:11:38.676110 23840 leader_election.cc:288] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [CANDIDATE]: Term 1 election: Requested vote from peers 72623fccae984f90b9dacc70042ff743 (127.12.212.3:36643), 4d4362283c7b4a8e82cff9984cfc63c8 (127.12.212.1:43293)
I20260924 23:11:38.676337 23769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0222a1a6d66640e38cb5b274c7e7d844" candidate_uuid: "977a59caf42e4be4866756fba22de467" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72623fccae984f90b9dacc70042ff743"
I20260924 23:11:38.676306 23501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0222a1a6d66640e38cb5b274c7e7d844" candidate_uuid: "977a59caf42e4be4866756fba22de467" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4362283c7b4a8e82cff9984cfc63c8"
I20260924 23:11:38.676429 23769 raft_consensus.cc:3037] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:38.676463 23501 raft_consensus.cc:3037] T 0222a1a6d66640e38cb5b274c7e7d844 P 4d4362283c7b4a8e82cff9984cfc63c8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:11:38.677338 23769 raft_consensus.cc:2454] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977a59caf42e4be4866756fba22de467 in term 1.
I20260924 23:11:38.677337 23501 raft_consensus.cc:2454] T 0222a1a6d66640e38cb5b274c7e7d844 P 4d4362283c7b4a8e82cff9984cfc63c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977a59caf42e4be4866756fba22de467 in term 1.
I20260924 23:11:38.677547 23568 leader_election.cc:302] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d4362283c7b4a8e82cff9984cfc63c8, 977a59caf42e4be4866756fba22de467; no voters:
I20260924 23:11:38.677654 23840 raft_consensus.cc:2781] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:11:38.677837 23840 raft_consensus.cc:686] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [term 1 LEADER]: Becoming Leader. State: Replica: 977a59caf42e4be4866756fba22de467, State: Running, Role: LEADER
I20260924 23:11:38.677944 23840 consensus_queue.cc:237] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72623fccae984f90b9dacc70042ff743" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36643 } } peers { permanent_uuid: "977a59caf42e4be4866756fba22de467" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34531 } } peers { permanent_uuid: "4d4362283c7b4a8e82cff9984cfc63c8" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 43293 } }
I20260924 23:11:38.678992 23355 catalog_manager.cc:5495] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 reported cstate change: term changed from 0 to 1, leader changed from <none> to 977a59caf42e4be4866756fba22de467 (127.12.212.2). New cstate: current_term: 1 leader_uuid: "977a59caf42e4be4866756fba22de467" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72623fccae984f90b9dacc70042ff743" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977a59caf42e4be4866756fba22de467" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4362283c7b4a8e82cff9984cfc63c8" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 43293 } health_report { overall_health: UNKNOWN } } }
I20260924 23:11:38.705668 23830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:38.703091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44840 (local address 127.12.212.2:34531)
0924 23:11:38.703168 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 23:11:38.703170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:11:38.703179 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:11:38.703219 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:11:38.703220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:11:38.703239 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:11:38.703301 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:11:38.703506 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.704007 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.704816 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:11:38.705030 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:11:38.705099 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:11:38.705239 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:11:38.705391 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:11:38.705461 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 23:11:38.705535 (+ 74us) server_negotiation.cc:299] Negotiation successful
0924 23:11:38.705558 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 23:11:38.707598 23769 raft_consensus.cc:1261] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743 [term 1 FOLLOWER]: Refusing update from remote peer 977a59caf42e4be4866756fba22de467: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:38.707598 23501 raft_consensus.cc:1261] T 0222a1a6d66640e38cb5b274c7e7d844 P 4d4362283c7b4a8e82cff9984cfc63c8 [term 1 FOLLOWER]: Refusing update from remote peer 977a59caf42e4be4866756fba22de467: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:11:38.707962 23840 consensus_queue.cc:1035] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4362283c7b4a8e82cff9984cfc63c8" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 43293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:38.708122 23842 consensus_queue.cc:1035] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72623fccae984f90b9dacc70042ff743" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:11:38.713495 23355 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:11:38.713593 23355 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:11:38.714347 23355 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b610eee9ba72400caad499003ca9ed0a: Sending DeleteTablet for 3 replicas of tablet 0222a1a6d66640e38cb5b274c7e7d844
I20260924 23:11:38.714659 23615 tablet_service.cc:1514] Processing DeleteTablet for tablet 0222a1a6d66640e38cb5b274c7e7d844 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:38 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:44830
I20260924 23:11:38.714681 23481 tablet_service.cc:1514] Processing DeleteTablet for tablet 0222a1a6d66640e38cb5b274c7e7d844 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:38 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:37454
I20260924 23:11:38.714934 23749 tablet_service.cc:1514] Processing DeleteTablet for tablet 0222a1a6d66640e38cb5b274c7e7d844 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:11:38 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:56092
I20260924 23:11:38.715196 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 23418
I20260924 23:11:38.715122 23857 tablet_replica.cc:330] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743: stopping tablet replica
I20260924 23:11:38.715864 23856 tablet_replica.cc:330] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467: stopping tablet replica
I20260924 23:11:38.715958 23856 raft_consensus.cc:2229] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:11:38.715960 23857 raft_consensus.cc:2229] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:11:38.716058 23856 raft_consensus.cc:2258] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:38.716063 23857 raft_consensus.cc:2258] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:11:38.716567 23857 ts_tablet_manager.cc:1905] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:38.716568 23856 ts_tablet_manager.cc:1905] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:11:38.718397 23857 ts_tablet_manager.cc:1918] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:11:38.718398 23856 ts_tablet_manager.cc:1918] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:11:38.718488 23856 log.cc:1196] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/0222a1a6d66640e38cb5b274c7e7d844
I20260924 23:11:38.718488 23857 log.cc:1196] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/0222a1a6d66640e38cb5b274c7e7d844
I20260924 23:11:38.718804 23856 ts_tablet_manager.cc:1939] T 0222a1a6d66640e38cb5b274c7e7d844 P 977a59caf42e4be4866756fba22de467: Deleting consensus metadata
I20260924 23:11:38.719033 23857 ts_tablet_manager.cc:1939] T 0222a1a6d66640e38cb5b274c7e7d844 P 72623fccae984f90b9dacc70042ff743: Deleting consensus metadata
I20260924 23:11:38.719606 23343 catalog_manager.cc:4841] TS 72623fccae984f90b9dacc70042ff743 (127.12.212.3:36643): tablet 0222a1a6d66640e38cb5b274c7e7d844 (table test-table [id=06703e8ba9a3462db904d1daaa2cef60]) successfully deleted
I20260924 23:11:38.719600 23340 catalog_manager.cc:4841] TS 977a59caf42e4be4866756fba22de467 (127.12.212.2:34531): tablet 0222a1a6d66640e38cb5b274c7e7d844 (table test-table [id=06703e8ba9a3462db904d1daaa2cef60]) successfully deleted
W20260924 23:11:38.721185 23340 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.212.1:43293 (error 108)
I20260924 23:11:38.721402 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 23552
I20260924 23:11:38.722051 23827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:11:38.721593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:43293 (local address 127.0.0.1:37468)
0924 23:11:38.721965 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0924 23:11:38.722015 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.1:43293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351}
W20260924 23:11:38.722393 23340 catalog_manager.cc:4584] TS 4d4362283c7b4a8e82cff9984cfc63c8 (127.12.212.1:43293): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0222a1a6d66640e38cb5b274c7e7d844: Network error: Client connection negotiation failed: client connection to 127.12.212.1:43293: connect: Connection refused (error 111)
I20260924 23:11:38.725893 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 23686
I20260924 23:11:38.731297 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 23324
2026-09-24T23:11:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1963 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:11:38 dist-test-slave-m0z6 krb5kdc[23861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:11:38 dist-test-slave-m0z6 krb5kdc[23861](info): set up 2 sockets
Sep 24 23:11:38 dist-test-slave-m0z6 krb5kdc[23861](info): commencing operation
krb5kdc: starting...
W20260924 23:11:39.691390 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:11:39 dist-test-slave-m0z6 krb5kdc[23861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:11:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:11:39Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.12.212.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.12.212.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.12.212.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/rangeradmin_127.12.212.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.12.212.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/rangeradmin_127.12.212.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.12.212.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.12.212.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.12.212.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/rangerlookup_127.12.212.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.12.212.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/rangerlookup_127.12.212.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.212.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.12.212.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.212.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/HTTP_127.12.212.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.212.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/HTTP_127.12.212.20@KRBTEST.COM.keytab.
I20260924 23:11:39.857609 13136 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskmL_OPf/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/postgres -l logfile start
2026-09-24 23:11:42.472 UTC [23891] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-24 23:11:42.472 UTC [23891] LOG: listening on IPv4 address "127.12.212.20", port 57901
2026-09-24 23:11:42.475 UTC [23891] LOG: listening on Unix socket "/tmp/.s.PGSQL.57901"
2026-09-24 23:11:42.495 UTC [23894] LOG: database system was shut down at 2026-09-24 23:11:40 UTC
2026-09-24 23:11:42.498 UTC [23891] LOG: database system is ready to accept connections
I20260924 23:11:43.367282 13136 mini_postgres.cc:96] Postgres bound to 57901
2026-09-24 23:11:43.373 UTC [23902] FATAL: database "slave" does not exist
127.12.212.20:57901 - accepting connections
I20260924 23:11:43.373579 13136 mini_ranger.cc:161] Starting Ranger...
I20260924 23:11:43.387399 13136 mini_ranger.cc:84] Created miniranger Postgres user
I20260924 23:11:43.912979 13136 mini_ranger.cc:87] Created ranger Postgres database
I20260924 23:11:43.913060 13136 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ranger-admin
2026-09-24 23:11:44,187 [I] DB FLAVOR :POSTGRES
2026-09-24 23:11:44,187 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 23:11:44,187 [I] Checking connection..
2026-09-24 23:11:44,188 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-24 23:11:44,438 [I] Checking connection passed.
2026-09-24 23:11:44,438 [I] --------- Verifying version history table ---------
2026-09-24 23:11:44,439 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 23:11:44,686 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 23:11:44,687 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 23:11:44,938 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 23:11:44,938 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-24 23:11:44,938 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-24 23:11:45.201 UTC [23966] WARNING: there is no transaction in progress
2026-09-24 23:11:45,207 [I] create_dbversion_catalog.sql file imported successfully
2026-09-24 23:11:45,207 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 23:11:45,444 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-24 23:11:45,444 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 23:11:45,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-24 23:11:45,693 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-24 23:11:45,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-m0z6.c.gcp-upstream.internal','N') ;"
2026-09-24 23:11:46,187 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 23:11:46,455 [I] Table x_portal_user does not exist in database ranger
2026-09-24 23:11:46,455 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 23:11:46,716 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-24 23:11:46,716 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 23:11:46,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 23:11:47,198 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-24 23:11:47,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-24 23:11:48.193 UTC [24074] WARNING: there is no transaction in progress
2026-09-24 23:11:48.214 UTC [24074] WARNING: there is no transaction in progress
2026-09-24 23:11:48.232 UTC [24074] WARNING: there is no transaction in progress
2026-09-24 23:11:48.249 UTC [24074] WARNING: there is no transaction in progress
2026-09-24 23:11:48.267 UTC [24074] WARNING: there is no transaction in progress
2026-09-24 23:11:48.287 UTC [24074] WARNING: there is no transaction in progress
2026-09-24 23:11:48.353 UTC [24074] WARNING: there is no transaction in progress
2026-09-24 23:11:48.362 UTC [24074] WARNING: there is no transaction in progress
2026-09-24 23:11:48.379 UTC [24074] WARNING: there is no transaction in progress
2026-09-24 23:11:48.392 UTC [24074] WARNING: there is no transaction in progress
2026-09-24 23:11:48.403 UTC [24074] WARNING: there is no transaction in progress
2026-09-24 23:11:48.418 UTC [24074] WARNING: there is no transaction in progress
2026-09-24 23:11:48.430 UTC [24074] WARNING: there is no transaction in progress
2026-09-24 23:11:48.438 UTC [24074] WARNING: there is no transaction in progress
2026-09-24 23:11:48.446 UTC [24074] WARNING: there is no transaction in progress
2026-09-24 23:11:48.453 UTC [24074] WARNING: there is no transaction in progress
2026-09-24 23:11:49,220 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-24 23:11:49,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-24 23:11:49,480 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-24 23:11:49,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 23:11:49,719 [I] Table x_portal_user already exists in database 'ranger'
2026-09-24 23:11:49,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 23:11:49,975 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-24 23:11:49,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 23:11:50,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 23:11:50,479 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-m0z6.c.gcp-upstream.internal';"
2026-09-24 23:11:50,718 [I] CORE_DB_SCHEMA import status has been updated
2026-09-24 23:11:50,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-24 23:11:50,961 [I] DB_PATCHES have already been applied
I20260924 23:11:50.966871 13136 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ranger-admin:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskmL_OPf/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260924 23:11:50.966944 13136 mini_ranger.cc:193] Using host: 127.12.212.20
I20260924 23:11:50.969982 13136 mini_ranger.cc:227] Ranger admin URL: http://127.12.212.20:54673
Sep 24, 2026 11:11:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 11:11:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 11:11:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ranger-admin/ews/webapp
Sep 24, 2026 11:11:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 24, 2026 11:11:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 24, 2026 11:11:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ranger-admin/ews/webapp].
Sep 24, 2026 11:11:51 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260924 23:11:51.952394 13136 mini_ranger.cc:160] Time spent starting Ranger: real 8.579s user 0.000s sys 0.009s
Sep 24, 2026 11:12:13 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 24, 2026 11:12:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 24, 2026 11:12:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 24, 2026 11:12:13 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 24, 2026 11:12:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 24, 2026 11:12:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 24, 2026 11:12:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 24, 2026 11:12:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 24, 2026 11:12:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 24, 2026 11:12:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 24, 2026 11:12:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 24, 2026 11:12:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 24, 2026 11:12:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 24, 2026 11:12:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 24, 2026 11:12:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 24, 2026 11:12:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 24, 2026 11:12:13 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 24, 2026 11:12:14 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260924 23:12:15.365818 13136 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.12.212.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.12.212.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.12.212.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/rangerkms_127.12.212.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.12.212.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/rangerkms_127.12.212.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.212.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.12.212.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.212.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/HTTP_127.12.212.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.212.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/HTTP_127.12.212.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 24 23:12:15 dist-test-slave-m0z6 krb5kdc[23861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291535, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260924 23:12:15.456718 13136 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260924 23:12:15.466380 13136 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260924 23:12:16.480525 13136 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260924 23:12:16.480600 13136 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ranger-kms
I20260924 23:12:16.480612 13136 mini_ranger_kms.cc:227] Using postgres at 127.12.212.20:57901
2026-09-24 23:12:16,631 [I] DB FLAVOR :POSTGRES
2026-09-24 23:12:16,632 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 23:12:16,632 [I] Checking connection
2026-09-24 23:12:16,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-24 23:12:16,914 [I] connection success
2026-09-24 23:12:16,914 [I] --------- Verifying Ranger DB tables ---------
2026-09-24 23:12:16,914 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-24 23:12:16,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-24 23:12:17,162 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-24 23:12:17,162 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 23:12:17,162 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-24 23:12:17,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:57901/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-24 23:12:17,437 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260924 23:12:17.978766 13136 mini_ranger_kms.cc:310] Created kms service
I20260924 23:12:18.226106 13136 mini_ranger_kms.cc:326] Created kudu user
I20260924 23:12:18.273319 13136 mini_ranger_kms.cc:343] Created rangerkms user
Sep 24 23:12:18 dist-test-slave-m0z6 krb5kdc[23861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291535, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
I20260924 23:12:18.693327 13136 mini_ranger_kms.cc:384] Added ranger policy
I20260924 23:12:18.693455 13136 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ranger-kms:/tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260924 23:12:18.693475 13136 mini_ranger_kms.cc:240] Using host: 127.12.212.20
I20260924 23:12:18.697188 13136 mini_ranger_kms.cc:276] Ranger KMS PID: 24255
I20260924 23:12:18.697319 13136 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.12.212.20:46045
Sep 24, 2026 11:12:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 11:12:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 11:12:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 24, 2026 11:12:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 24, 2026 11:12:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ranger-kms/ews/webapp].
Sep 24, 2026 11:12:20 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260924 23:12:20.778232 13136 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.322s user 0.011s sys 0.122s
I20260924 23:12:20.778354 13136 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260924 23:12:20.778460 13136 mini_ranger_kms.cc:401] 127.12.212.20:46045/kms/v1/keys
Sep 24 23:12:20 dist-test-slave-m0z6 krb5kdc[23861](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790291540, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.212.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 24 23:12:21 dist-test-slave-m0z6 krb5kdc[23861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291540, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.212.20@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
Sep 24 23:12:21 dist-test-slave-m0z6 krb5kdc[23861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291540, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.212.20@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
Sep 24 23:12:24 dist-test-slave-m0z6 krb5kdc[23861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:12:24.169894 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:36605
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:36605
--ranger_config_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.212.20:46045/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:12:24.251454 24305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:12:24.251715 24305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:12:24.251739 24305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:12:24.252964 24305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:12:24.253014 24305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:12:24.253028 24305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:12:24.253039 24305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:12:24.253050 24305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:12:24.254763 24305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/master-0/wal
--ranger_kms_url=127.12.212.20:46045/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:36605
--ranger_config_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:36605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:12:24.254997 24305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:12:24.255229 24305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:12:24.260519 24313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:12:24.260535 24311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:12:24.260577 24305 server_base.cc:1034] running on GCE node
W20260924 23:12:24.260649 24310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:12:24.261098 24305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:12:24.261417 24305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:12:24.262603 24305 hybrid_clock.cc:648] HybridClock initialized: now 1790291544262584 us; error 39 us; skew 500 ppm
Sep 24 23:12:24 dist-test-slave-m0z6 krb5kdc[23861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291544, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:12:24.271952 24305 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:12:24.272548 24305 webserver.cc:460] Webserver started at http://127.12.212.62:41543/ using document root <none> and password file <none>
I20260924 23:12:24.272773 24305 fs_manager.cc:359] Metadata directory not provided
I20260924 23:12:24.272817 24305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:12:24.272975 24305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 23:12:24 dist-test-slave-m0z6 krb5kdc[23861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291544, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
I20260924 23:12:24.321563 24305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "a0de329b221847849db3051fd5971ae6"
format_stamp: "Formatted at 2026-09-24 23:12:24 on dist-test-slave-m0z6"
server_key: "2525b1d037c6f4cdfa7914bb960dd97f"
server_key_iv: "0b4227dd4cd01bc46344e9769dece76b"
server_key_version: "kuduclusterkey@0"
I20260924 23:12:24.322049 24305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "a0de329b221847849db3051fd5971ae6"
format_stamp: "Formatted at 2026-09-24 23:12:24 on dist-test-slave-m0z6"
server_key: "2525b1d037c6f4cdfa7914bb960dd97f"
server_key_iv: "0b4227dd4cd01bc46344e9769dece76b"
server_key_version: "kuduclusterkey@0"
I20260924 23:12:24.323992 24305 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260924 23:12:24.340677 24320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:12:24.341348 24305 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:12:24.341480 24305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "a0de329b221847849db3051fd5971ae6"
format_stamp: "Formatted at 2026-09-24 23:12:24 on dist-test-slave-m0z6"
server_key: "2525b1d037c6f4cdfa7914bb960dd97f"
server_key_iv: "0b4227dd4cd01bc46344e9769dece76b"
server_key_version: "kuduclusterkey@0"
I20260924 23:12:24.341596 24305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:12:24.350936 24305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:12:24.352903 24305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:12:24.353052 24305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:12:24.357789 24305 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:36605
I20260924 23:12:24.357803 24372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:36605 every 8 connection(s)
I20260924 23:12:24.358209 24305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:12:24.359055 24373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:12:24.362335 24373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6: Bootstrap starting.
I20260924 23:12:24.363209 24373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6: Neither blocks nor log segments found. Creating new log.
I20260924 23:12:24.363561 24373 log.cc:824] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6: Log is configured to *not* fsync() on all Append() calls
I20260924 23:12:24.364410 24373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6: No bootstrap required, opened a new log
I20260924 23:12:24.365864 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 24305
I20260924 23:12:24.365998 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:12:24.366178 13136 external_mini_cluster.cc:1428] Setting key 0f0f9bfa1decdee7d0533e91bc27f355
I20260924 23:12:24.367580 24373 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0de329b221847849db3051fd5971ae6" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36605 } }
I20260924 23:12:24.367899 24373 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:12:24.367957 24373 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0de329b221847849db3051fd5971ae6, State: Initialized, Role: FOLLOWER
I20260924 23:12:24.368115 24373 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0de329b221847849db3051fd5971ae6" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36605 } }
I20260924 23:12:24.368198 24373 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:12:24.368227 24373 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:12:24.368294 24373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:12:24.369120 24373 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0de329b221847849db3051fd5971ae6" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36605 } }
I20260924 23:12:24.369237 24373 leader_election.cc:302] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0de329b221847849db3051fd5971ae6; no voters:
I20260924 23:12:24.369448 24373 leader_election.cc:288] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:12:24.369503 24378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:12:24.369735 24378 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6 [term 1 LEADER]: Becoming Leader. State: Replica: a0de329b221847849db3051fd5971ae6, State: Running, Role: LEADER
I20260924 23:12:24.369812 24373 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:12:24.369920 24378 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0de329b221847849db3051fd5971ae6" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36605 } }
I20260924 23:12:24.370319 24378 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0de329b221847849db3051fd5971ae6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0de329b221847849db3051fd5971ae6" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36605 } } }
I20260924 23:12:24.370453 24378 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6 [sys.catalog]: This master's current role is: LEADER
I20260924 23:12:24.370627 24378 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0de329b221847849db3051fd5971ae6. Latest consensus state: current_term: 1 leader_uuid: "a0de329b221847849db3051fd5971ae6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0de329b221847849db3051fd5971ae6" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36605 } } }
I20260924 23:12:24.370751 24378 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6 [sys.catalog]: This master's current role is: LEADER
I20260924 23:12:24.370905 24387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:12:24.371167 24373 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 23:12:24.371451 24387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 23:12:24 dist-test-slave-m0z6 krb5kdc[23861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:12:24.377923 24396 mvcc.cc:205] Tried to move back new op lower bound from 7333034165764501504 to 7333034165739855872. Current Snapshot: MvccSnapshot[applied={T|T < 7333034165764501504}]
I20260924 23:12:24.381716 24377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:24.366681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56370 (local address 127.12.212.62:36605)
0924 23:12:24.366888 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0924 23:12:24.366899 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 23:12:24.366930 (+ 31us) server_negotiation.cc:407] Connection header received
0924 23:12:24.367503 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:12:24.367510 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:12:24.367570 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:12:24.367736 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:12:24.368438 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:24.369472 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:24.370532 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:24.371184 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:24.371288 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:12:24.374222 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:12:24.374278 (+ 56us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:12:24.374278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:12:24.374374 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:12:24.379156 (+ 4782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:12:24.379587 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:12:24.379590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:12:24.379590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:12:24.380128 (+ 538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:12:24.380341 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:12:24.380342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:12:24.380343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:12:24.380686 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:12:24.381259 (+ 573us) server_negotiation.cc:1035] Waiting for connection context
0924 23:12:24.381339 (+ 80us) server_negotiation.cc:299] Negotiation successful
0924 23:12:24.381498 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":83,"threads_started":1}
I20260924 23:12:24.383160 24387 catalog_manager.cc:1347] Generated new cluster ID: bee6cde64e854ba8a3e588b1ae77ea9e
I20260924 23:12:24.383217 24387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:12:24.392781 24387 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:12:24.393631 24387 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:12:24.401557 24387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6: Generated new TSK 0
I20260924 23:12:24.401886 24387 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
I20260924 23:12:25.182137 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.212.20:46045/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:36605
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:12:25.301302 24418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:12:25.301539 24418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:12:25.301571 24418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:12:25.303021 24418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:12:25.303076 24418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:12:25.303115 24418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:12:25.304791 24418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.12.212.20:46045/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:36605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:12:25.305085 24418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:12:25.305336 24418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:12:25.310312 24426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:12:25.310411 24424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:12:25.310316 24423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:12:25.311123 24418 server_base.cc:1034] running on GCE node
I20260924 23:12:25.311374 24418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:12:25.311645 24418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:12:25.312878 24418 hybrid_clock.cc:648] HybridClock initialized: now 1790291545312803 us; error 37 us; skew 500 ppm
Sep 24 23:12:25 dist-test-slave-m0z6 krb5kdc[23861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291545, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:12:25.316249 24418 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:12:25.316836 24418 webserver.cc:460] Webserver started at http://127.12.212.1:44297/ using document root <none> and password file <none>
I20260924 23:12:25.317147 24418 fs_manager.cc:359] Metadata directory not provided
I20260924 23:12:25.317204 24418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:12:25.317332 24418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 23:12:25 dist-test-slave-m0z6 krb5kdc[23861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291545, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
I20260924 23:12:25.334028 24418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "85c5899db90b43528987738c42309e3f"
format_stamp: "Formatted at 2026-09-24 23:12:25 on dist-test-slave-m0z6"
server_key: "46c8f0945fcf46d4751ddc93db44b1b6"
server_key_iv: "86eef4f23a8191a28a60a8182453efb3"
server_key_version: "kuduclusterkey@0"
I20260924 23:12:25.334508 24418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "85c5899db90b43528987738c42309e3f"
format_stamp: "Formatted at 2026-09-24 23:12:25 on dist-test-slave-m0z6"
server_key: "46c8f0945fcf46d4751ddc93db44b1b6"
server_key_iv: "86eef4f23a8191a28a60a8182453efb3"
server_key_version: "kuduclusterkey@0"
I20260924 23:12:25.336412 24418 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260924 23:12:25.352005 24433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:12:25.352201 24418 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 23:12:25.352298 24418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "85c5899db90b43528987738c42309e3f"
format_stamp: "Formatted at 2026-09-24 23:12:25 on dist-test-slave-m0z6"
server_key: "46c8f0945fcf46d4751ddc93db44b1b6"
server_key_iv: "86eef4f23a8191a28a60a8182453efb3"
server_key_version: "kuduclusterkey@0"
I20260924 23:12:25.352407 24418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:12:25.390537 24418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:12:25.393288 24418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:12:25.393529 24418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:12:25.393867 24418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:12:25.394285 24418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:12:25.394333 24418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:12:25.394387 24418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:12:25.394424 24418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:12:25.401237 24418 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:46105
I20260924 23:12:25.401684 24418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:12:25.402139 24548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:46105 every 8 connection(s)
Sep 24 23:12:25 dist-test-slave-m0z6 krb5kdc[23861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291545, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:12:25.411430 24551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:25.402626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:55357 (local address 127.12.212.62:36605)
0924 23:12:25.402764 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0924 23:12:25.402766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:12:25.403061 (+ 295us) server_negotiation.cc:407] Connection header received
0924 23:12:25.403594 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:12:25.403595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:12:25.403610 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:12:25.403671 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:12:25.404230 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.404740 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.405745 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.405957 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.406021 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:12:25.408043 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:12:25.408044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:12:25.408045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:12:25.408053 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:12:25.409665 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:12:25.410224 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:12:25.410225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:12:25.410226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:12:25.410301 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:12:25.410612 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:12:25.410613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:12:25.410613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:12:25.410795 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:12:25.410867 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 23:12:25.411226 (+ 359us) server_negotiation.cc:299] Negotiation successful
0924 23:12:25.411321 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
I20260924 23:12:25.411618 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:25.402620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36605 (local address 127.12.212.1:55357)
0924 23:12:25.402927 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0924 23:12:25.402961 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 23:12:25.403495 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:12:25.403748 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:12:25.403751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:12:25.403939 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:12:25.404143 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:12:25.404147 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.404837 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.404838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:12:25.405662 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:12:25.405664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.406039 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.406040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:12:25.406161 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:12:25.406666 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:12:25.406675 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:12:25.407942 (+ 1267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:12:25.409766 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:12:25.409774 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:12:25.409777 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:12:25.410137 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:12:25.410379 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:12:25.410379 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:12:25.410380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:12:25.410534 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:12:25.410868 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:12:25.410874 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:12:25.411066 (+ 192us) client_negotiation.cc:769] Sending connection context
0924 23:12:25.411192 (+ 126us) client_negotiation.cc:240] Negotiation successful
0924 23:12:25.411374 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":145,"threads_started":1}
I20260924 23:12:25.412200 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 24418
I20260924 23:12:25.412309 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:12:25.412288 24549 heartbeater.cc:347] Connected to a master server at 127.12.212.62:36605
I20260924 23:12:25.412384 24549 heartbeater.cc:464] Registering TS with master...
I20260924 23:12:25.412513 13136 external_mini_cluster.cc:1428] Setting key 6ce2dabe75e56cfe5f37f6b9f16e9b9c
I20260924 23:12:25.412576 24549 heartbeater.cc:511] Master 127.12.212.62:36605 requested a full tablet report, sending...
I20260924 23:12:25.413273 24336 ts_manager.cc:194] Registered new tserver with Master: 85c5899db90b43528987738c42309e3f (127.12.212.1:46105)
I20260924 23:12:25.414203 24336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:55357
WARNING: no policy specified for kudu/127.12.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
I20260924 23:12:25.442476 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.212.20:46045/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:36605
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:12:25.565317 24555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:12:25.565623 24555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:12:25.565665 24555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:12:25.567624 24555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:12:25.567687 24555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
Sep 24 23:12:25 dist-test-slave-m0z6 krb5kdc[23861](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790291545, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260924 23:12:25.567728 24555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:12:25.570443 24555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.12.212.20:46045/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:36605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:12:25.570780 24555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:12:25.571117 24555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:12:25.578019 24563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:12:25.578359 24562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:12:25.578827 24555 server_base.cc:1034] running on GCE node
W20260924 23:12:25.579183 24565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:12:25.579557 24555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:12:25.579888 24555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:12:25.581084 24555 hybrid_clock.cc:648] HybridClock initialized: now 1790291545581069 us; error 29 us; skew 500 ppm
Sep 24 23:12:25 dist-test-slave-m0z6 krb5kdc[23861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291545, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:12:25.584620 24555 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:12:25.585292 24555 webserver.cc:460] Webserver started at http://127.12.212.2:36259/ using document root <none> and password file <none>
I20260924 23:12:25.585628 24555 fs_manager.cc:359] Metadata directory not provided
I20260924 23:12:25.585748 24555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:12:25.585933 24555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 23:12:25 dist-test-slave-m0z6 krb5kdc[23861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291545, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
I20260924 23:12:25.603861 24555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "15cb3997f8a64c0cadb7e7c35830bd13"
format_stamp: "Formatted at 2026-09-24 23:12:25 on dist-test-slave-m0z6"
server_key: "ef78869eaac5ff999fbd31f9be34da2c"
server_key_iv: "c8e8e4202f9dca989dab426f231a0b0e"
server_key_version: "kuduclusterkey@0"
I20260924 23:12:25.604763 24555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "15cb3997f8a64c0cadb7e7c35830bd13"
format_stamp: "Formatted at 2026-09-24 23:12:25 on dist-test-slave-m0z6"
server_key: "ef78869eaac5ff999fbd31f9be34da2c"
server_key_iv: "c8e8e4202f9dca989dab426f231a0b0e"
server_key_version: "kuduclusterkey@0"
I20260924 23:12:25.607214 24555 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260924 23:12:25.625932 24572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:12:25.626187 24555 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 23:12:25.626269 24555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "15cb3997f8a64c0cadb7e7c35830bd13"
format_stamp: "Formatted at 2026-09-24 23:12:25 on dist-test-slave-m0z6"
server_key: "ef78869eaac5ff999fbd31f9be34da2c"
server_key_iv: "c8e8e4202f9dca989dab426f231a0b0e"
server_key_version: "kuduclusterkey@0"
I20260924 23:12:25.626356 24555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:12:25.654596 24555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:12:25.657100 24555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:12:25.657253 24555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:12:25.657508 24555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:12:25.657886 24555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:12:25.657923 24555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:12:25.657951 24555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:12:25.657968 24555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:12:25.664287 24555 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:43277
I20260924 23:12:25.664788 24555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:12:25.666405 24686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:43277 every 8 connection(s)
I20260924 23:12:25.670904 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 24555
I20260924 23:12:25.671037 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:12:25.671232 13136 external_mini_cluster.cc:1428] Setting key c552acb480efd5b3b5971bd3941ef006
Sep 24 23:12:25 dist-test-slave-m0z6 krb5kdc[23861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291545, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:12:25.680665 24551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:25.668373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:58575 (local address 127.12.212.62:36605)
0924 23:12:25.668454 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0924 23:12:25.668457 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:12:25.669733 (+ 1276us) server_negotiation.cc:407] Connection header received
0924 23:12:25.670581 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:12:25.670583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:12:25.670602 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:12:25.670634 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:12:25.671882 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.672665 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.674249 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.674528 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.674571 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:12:25.677344 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:12:25.677346 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:12:25.677346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:12:25.677355 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:12:25.679012 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:12:25.679528 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:12:25.679530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:12:25.679530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:12:25.679589 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:12:25.679851 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:12:25.679852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:12:25.679853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:12:25.680002 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:12:25.680043 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 23:12:25.680453 (+ 410us) server_negotiation.cc:299] Negotiation successful
0924 23:12:25.680558 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 23:12:25.681509 24689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:25.668062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36605 (local address 127.12.212.2:58575)
0924 23:12:25.669533 (+ 1471us) negotiation.cc:107] Waiting for socket to connect
0924 23:12:25.669604 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0924 23:12:25.670426 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:12:25.670897 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:12:25.670903 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:12:25.671412 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:12:25.671736 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:12:25.671754 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.672790 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.672791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:12:25.674145 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:12:25.674148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.674667 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.674668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:12:25.674846 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:12:25.675523 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:12:25.675534 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:12:25.677220 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:12:25.679111 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:12:25.679120 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:12:25.679123 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:12:25.679442 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:12:25.679654 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:12:25.679655 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:12:25.679655 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:12:25.679789 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:12:25.680087 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:12:25.680094 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:12:25.680285 (+ 191us) client_negotiation.cc:769] Sending connection context
0924 23:12:25.681058 (+ 773us) client_negotiation.cc:240] Negotiation successful
0924 23:12:25.681250 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1347,"thread_start_us":99,"threads_started":1}
I20260924 23:12:25.682382 24687 heartbeater.cc:347] Connected to a master server at 127.12.212.62:36605
I20260924 23:12:25.682490 24687 heartbeater.cc:464] Registering TS with master...
I20260924 23:12:25.682789 24687 heartbeater.cc:511] Master 127.12.212.62:36605 requested a full tablet report, sending...
I20260924 23:12:25.683218 24336 ts_manager.cc:194] Registered new tserver with Master: 15cb3997f8a64c0cadb7e7c35830bd13 (127.12.212.2:43277)
I20260924 23:12:25.683730 24336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:58575
WARNING: no policy specified for kudu/127.12.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
I20260924 23:12:25.713605 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.212.20:46045/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:36605
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:12:25.810834 24692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:12:25.811192 24692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:12:25.811267 24692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:12:25.813189 24692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:12:25.813318 24692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:12:25.813391 24692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:12:25.816200 24692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.12.212.20:46045/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:36605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:12:25.816718 24692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:12:25.817073 24692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:12:25.824263 24697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:12:25.824572 24698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:12:25.825291 24700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:12:25.826146 24692 server_base.cc:1034] running on GCE node
I20260924 23:12:25.826390 24692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:12:25.826656 24692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:12:25.827864 24692 hybrid_clock.cc:648] HybridClock initialized: now 1790291545827848 us; error 28 us; skew 500 ppm
Sep 24 23:12:25 dist-test-slave-m0z6 krb5kdc[23861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291545, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:12:25.831039 24692 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:12:25.831578 24692 webserver.cc:460] Webserver started at http://127.12.212.3:37999/ using document root <none> and password file <none>
I20260924 23:12:25.831887 24692 fs_manager.cc:359] Metadata directory not provided
I20260924 23:12:25.831981 24692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:12:25.832139 24692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 23:12:25 dist-test-slave-m0z6 krb5kdc[23861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291545, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
I20260924 23:12:25.852438 24692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "c003e06bdebe4332828e1c10f60f3367"
format_stamp: "Formatted at 2026-09-24 23:12:25 on dist-test-slave-m0z6"
server_key: "f411648023b25cff33ab87b80c0d74cc"
server_key_iv: "a58f50a829a4d031e812f91302f328f3"
server_key_version: "kuduclusterkey@0"
I20260924 23:12:25.852957 24692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "c003e06bdebe4332828e1c10f60f3367"
format_stamp: "Formatted at 2026-09-24 23:12:25 on dist-test-slave-m0z6"
server_key: "f411648023b25cff33ab87b80c0d74cc"
server_key_iv: "a58f50a829a4d031e812f91302f328f3"
server_key_version: "kuduclusterkey@0"
I20260924 23:12:25.857123 24692 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260924 23:12:25.874856 24707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:12:25.875103 24692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260924 23:12:25.875214 24692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "c003e06bdebe4332828e1c10f60f3367"
format_stamp: "Formatted at 2026-09-24 23:12:25 on dist-test-slave-m0z6"
server_key: "f411648023b25cff33ab87b80c0d74cc"
server_key_iv: "a58f50a829a4d031e812f91302f328f3"
server_key_version: "kuduclusterkey@0"
I20260924 23:12:25.875321 24692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:12:25.890282 24692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:12:25.896770 24692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:12:25.897025 24692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:12:25.897372 24692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:12:25.897934 24692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:12:25.898034 24692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:12:25.898113 24692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:12:25.898169 24692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:12:25.904758 24692 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:33017
I20260924 23:12:25.905434 24692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:12:25.914990 24820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:33017 every 8 connection(s)
I20260924 23:12:25.915556 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 24692
I20260924 23:12:25.915676 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:12:25.915879 13136 external_mini_cluster.cc:1428] Setting key de3b4eaa099876d51981ad9226275ee6
Sep 24 23:12:25 dist-test-slave-m0z6 krb5kdc[23861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291545, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:12:25.936151 24551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:25.917871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:56077 (local address 127.12.212.62:36605)
0924 23:12:25.918084 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0924 23:12:25.918086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:12:25.923078 (+ 4992us) server_negotiation.cc:407] Connection header received
0924 23:12:25.924075 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:12:25.924077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:12:25.924094 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:12:25.924127 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:12:25.925194 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.925938 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.927556 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.927868 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.927959 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:12:25.932508 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:12:25.932510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:12:25.932510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:12:25.932520 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:12:25.934236 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:12:25.934821 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:12:25.934822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:12:25.934823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:12:25.934897 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:12:25.935301 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:12:25.935302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:12:25.935303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:12:25.935462 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:12:25.935535 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 23:12:25.935942 (+ 407us) server_negotiation.cc:299] Negotiation successful
0924 23:12:25.936047 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260924 23:12:25.936349 24823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:25.915818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36605 (local address 127.12.212.3:56077)
0924 23:12:25.922871 (+ 7053us) negotiation.cc:107] Waiting for socket to connect
0924 23:12:25.922953 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0924 23:12:25.923884 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:12:25.924370 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:12:25.924375 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:12:25.924679 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:12:25.925023 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:12:25.925031 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.926079 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.926080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:12:25.927441 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:12:25.927445 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.927999 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.928000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:12:25.928177 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:12:25.929710 (+ 1533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:12:25.929722 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:12:25.932375 (+ 2653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:12:25.934345 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:12:25.934355 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:12:25.934359 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:12:25.934717 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:12:25.934996 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:12:25.934997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:12:25.934997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:12:25.935184 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:12:25.935548 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:12:25.935558 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:12:25.935762 (+ 204us) client_negotiation.cc:769] Sending connection context
0924 23:12:25.935884 (+ 122us) client_negotiation.cc:240] Negotiation successful
0924 23:12:25.936088 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6982,"thread_start_us":102,"threads_started":1}
I20260924 23:12:25.936971 24821 heartbeater.cc:347] Connected to a master server at 127.12.212.62:36605
I20260924 23:12:25.937100 24821 heartbeater.cc:464] Registering TS with master...
I20260924 23:12:25.937328 24821 heartbeater.cc:511] Master 127.12.212.62:36605 requested a full tablet report, sending...
I20260924 23:12:25.937744 24336 ts_manager.cc:194] Registered new tserver with Master: c003e06bdebe4332828e1c10f60f3367 (127.12.212.3:33017)
I20260924 23:12:25.938251 24336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:56077
I20260924 23:12:25.940946 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 23:12:25.954846 24551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:25.942979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56378 (local address 127.12.212.62:36605)
0924 23:12:25.943031 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 23:12:25.943033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:12:25.946934 (+ 3901us) server_negotiation.cc:407] Connection header received
0924 23:12:25.946997 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:12:25.946998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:12:25.947014 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:12:25.947090 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:12:25.947866 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.948656 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.949761 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.950061 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.950145 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:12:25.951757 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:12:25.951759 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:12:25.951759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:12:25.951769 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:12:25.953460 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:12:25.953897 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:12:25.953899 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:12:25.953900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:12:25.953967 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:12:25.954184 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:12:25.954185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:12:25.954185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:12:25.954320 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:12:25.954395 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 23:12:25.954611 (+ 216us) server_negotiation.cc:299] Negotiation successful
0924 23:12:25.954743 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 23:12:25.957221 24336 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:12:25.959637 24336 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:12:25.970546 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:25.964383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41204 (local address 127.12.212.2:43277)
0924 23:12:25.964792 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0924 23:12:25.964799 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 23:12:25.965110 (+ 311us) server_negotiation.cc:407] Connection header received
0924 23:12:25.965156 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:12:25.965159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:12:25.965175 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:12:25.965242 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:12:25.965495 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.966652 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.968716 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.969493 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.969565 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:12:25.969648 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 23:12:25.970416 (+ 768us) server_negotiation.cc:299] Negotiation successful
0924 23:12:25.970464 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":83,"threads_started":1}
I20260924 23:12:25.970984 24835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:25.964087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:43277 (local address 127.0.0.1:41204)
0924 23:12:25.964643 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0924 23:12:25.964925 (+ 282us) client_negotiation.cc:174] Beginning negotiation
0924 23:12:25.965019 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:12:25.965329 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:12:25.965333 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:12:25.965335 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:12:25.965409 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:12:25.965411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.966809 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.966810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:12:25.968590 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:12:25.968593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.969681 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.969682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:12:25.969767 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:12:25.969784 (+ 17us) client_negotiation.cc:769] Sending connection context
0924 23:12:25.969863 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 23:12:25.969924 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":8,"thread_start_us":109,"threads_started":1}
I20260924 23:12:25.971972 24618 tablet_service.cc:1467] Processing CreateTablet for tablet 6f559286975f4ba5a3a238a851057d10 (DEFAULT_TABLE table=test-table [id=d10789cf0f354ab8ac71ceef28937188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:12:25.972177 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:25.964483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37676 (local address 127.12.212.1:46105)
0924 23:12:25.964861 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0924 23:12:25.964867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 23:12:25.965619 (+ 752us) server_negotiation.cc:407] Connection header received
0924 23:12:25.965658 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:12:25.965660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:12:25.965676 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:12:25.965746 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:12:25.965942 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.967075 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.969396 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.970258 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.970376 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:12:25.970442 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 23:12:25.972029 (+ 1587us) server_negotiation.cc:299] Negotiation successful
0924 23:12:25.972089 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":56,"threads_started":1}
I20260924 23:12:25.972398 24618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f559286975f4ba5a3a238a851057d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:12:25.972457 24834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:25.964096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:46105 (local address 127.0.0.1:37676)
0924 23:12:25.965516 (+ 1420us) negotiation.cc:107] Waiting for socket to connect
0924 23:12:25.965525 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:12:25.965566 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:12:25.965820 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:12:25.965820 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:12:25.965822 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:12:25.965870 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:12:25.965871 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.967173 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.967174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:12:25.969297 (+ 2123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:12:25.969301 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.970635 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.970636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:12:25.970737 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:12:25.970741 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:12:25.970790 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 23:12:25.972380 (+ 1590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1402,"thread_start_us":112,"threads_started":1}
I20260924 23:12:25.973316 24465 tablet_service.cc:1467] Processing CreateTablet for tablet 6f559286975f4ba5a3a238a851057d10 (DEFAULT_TABLE table=test-table [id=d10789cf0f354ab8ac71ceef28937188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:12:25.973691 24465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f559286975f4ba5a3a238a851057d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:12:25.981840 24839 tablet_bootstrap.cc:492] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f: Bootstrap starting.
I20260924 23:12:25.982975 24839 tablet_bootstrap.cc:654] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f: Neither blocks nor log segments found. Creating new log.
I20260924 23:12:25.983464 24839 log.cc:824] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f: Log is configured to *not* fsync() on all Append() calls
I20260924 23:12:25.984438 24839 tablet_bootstrap.cc:492] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f: No bootstrap required, opened a new log
I20260924 23:12:25.984519 24839 ts_tablet_manager.cc:1397] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 23:12:25.984952 24833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:25.964359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:33017 (local address 127.0.0.1:48362)
0924 23:12:25.970492 (+ 6133us) negotiation.cc:107] Waiting for socket to connect
0924 23:12:25.970504 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:12:25.972300 (+ 1796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:12:25.978656 (+ 6356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:12:25.978658 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:12:25.978661 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:12:25.978759 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:12:25.978762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.981679 (+ 2917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.981681 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:12:25.982945 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:12:25.982951 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.984658 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.984659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:12:25.984731 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:12:25.984738 (+ 7us) client_negotiation.cc:769] Sending connection context
0924 23:12:25.984779 (+ 41us) client_negotiation.cc:240] Negotiation successful
0924 23:12:25.984856 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6104,"thread_start_us":59,"threads_started":1}
I20260924 23:12:25.986903 24840 tablet_bootstrap.cc:492] T 6f559286975f4ba5a3a238a851057d10 P 15cb3997f8a64c0cadb7e7c35830bd13: Bootstrap starting.
I20260924 23:12:25.987252 24838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:25.964617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48362 (local address 127.12.212.3:33017)
0924 23:12:25.967392 (+ 2775us) server_negotiation.cc:206] Beginning negotiation
0924 23:12:25.967398 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 23:12:25.978452 (+ 11054us) server_negotiation.cc:407] Connection header received
0924 23:12:25.978513 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:12:25.978516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:12:25.978536 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:12:25.978639 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:12:25.980834 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.981551 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.983624 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:25.984439 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:25.984524 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:12:25.984613 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 23:12:25.985141 (+ 528us) server_negotiation.cc:299] Negotiation successful
0924 23:12:25.985197 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2730,"thread_start_us":67,"threads_started":1}
I20260924 23:12:25.988257 24840 tablet_bootstrap.cc:654] T 6f559286975f4ba5a3a238a851057d10 P 15cb3997f8a64c0cadb7e7c35830bd13: Neither blocks nor log segments found. Creating new log.
I20260924 23:12:25.988225 24744 tablet_service.cc:1467] Processing CreateTablet for tablet 6f559286975f4ba5a3a238a851057d10 (DEFAULT_TABLE table=test-table [id=d10789cf0f354ab8ac71ceef28937188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:12:25.988546 24744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f559286975f4ba5a3a238a851057d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:12:25.988638 24840 log.cc:824] T 6f559286975f4ba5a3a238a851057d10 P 15cb3997f8a64c0cadb7e7c35830bd13: Log is configured to *not* fsync() on all Append() calls
I20260924 23:12:25.992144 24839 raft_consensus.cc:348] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15cb3997f8a64c0cadb7e7c35830bd13" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 43277 } } peers { permanent_uuid: "85c5899db90b43528987738c42309e3f" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46105 } } peers { permanent_uuid: "c003e06bdebe4332828e1c10f60f3367" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33017 } }
I20260924 23:12:25.992344 24839 raft_consensus.cc:374] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:12:25.992384 24839 raft_consensus.cc:729] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85c5899db90b43528987738c42309e3f, State: Initialized, Role: FOLLOWER
I20260924 23:12:25.992592 24839 consensus_queue.cc:260] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15cb3997f8a64c0cadb7e7c35830bd13" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 43277 } } peers { permanent_uuid: "85c5899db90b43528987738c42309e3f" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46105 } } peers { permanent_uuid: "c003e06bdebe4332828e1c10f60f3367" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33017 } }
I20260924 23:12:25.992894 24839 ts_tablet_manager.cc:1428] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260924 23:12:25.993026 24549 heartbeater.cc:503] Master 127.12.212.62:36605 was elected leader, sending a full tablet report...
I20260924 23:12:25.991755 24843 tablet_bootstrap.cc:492] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367: Bootstrap starting.
I20260924 23:12:25.995540 24843 tablet_bootstrap.cc:654] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367: Neither blocks nor log segments found. Creating new log.
I20260924 23:12:25.995935 24843 log.cc:824] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367: Log is configured to *not* fsync() on all Append() calls
I20260924 23:12:25.996867 24840 tablet_bootstrap.cc:492] T 6f559286975f4ba5a3a238a851057d10 P 15cb3997f8a64c0cadb7e7c35830bd13: No bootstrap required, opened a new log
I20260924 23:12:25.996867 24843 tablet_bootstrap.cc:492] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367: No bootstrap required, opened a new log
I20260924 23:12:25.996945 24843 ts_tablet_manager.cc:1397] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260924 23:12:25.997184 24840 ts_tablet_manager.cc:1397] T 6f559286975f4ba5a3a238a851057d10 P 15cb3997f8a64c0cadb7e7c35830bd13: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260924 23:12:25.999353 24843 raft_consensus.cc:348] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15cb3997f8a64c0cadb7e7c35830bd13" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 43277 } } peers { permanent_uuid: "85c5899db90b43528987738c42309e3f" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46105 } } peers { permanent_uuid: "c003e06bdebe4332828e1c10f60f3367" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33017 } }
I20260924 23:12:25.999571 24843 raft_consensus.cc:374] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:12:25.999614 24843 raft_consensus.cc:729] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c003e06bdebe4332828e1c10f60f3367, State: Initialized, Role: FOLLOWER
I20260924 23:12:25.999668 24840 raft_consensus.cc:348] T 6f559286975f4ba5a3a238a851057d10 P 15cb3997f8a64c0cadb7e7c35830bd13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15cb3997f8a64c0cadb7e7c35830bd13" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 43277 } } peers { permanent_uuid: "85c5899db90b43528987738c42309e3f" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46105 } } peers { permanent_uuid: "c003e06bdebe4332828e1c10f60f3367" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33017 } }
I20260924 23:12:25.999842 24840 raft_consensus.cc:374] T 6f559286975f4ba5a3a238a851057d10 P 15cb3997f8a64c0cadb7e7c35830bd13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:12:25.999915 24840 raft_consensus.cc:729] T 6f559286975f4ba5a3a238a851057d10 P 15cb3997f8a64c0cadb7e7c35830bd13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15cb3997f8a64c0cadb7e7c35830bd13, State: Initialized, Role: FOLLOWER
I20260924 23:12:25.999938 24843 consensus_queue.cc:260] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15cb3997f8a64c0cadb7e7c35830bd13" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 43277 } } peers { permanent_uuid: "85c5899db90b43528987738c42309e3f" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46105 } } peers { permanent_uuid: "c003e06bdebe4332828e1c10f60f3367" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33017 } }
I20260924 23:12:26.000099 24840 consensus_queue.cc:260] T 6f559286975f4ba5a3a238a851057d10 P 15cb3997f8a64c0cadb7e7c35830bd13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15cb3997f8a64c0cadb7e7c35830bd13" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 43277 } } peers { permanent_uuid: "85c5899db90b43528987738c42309e3f" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46105 } } peers { permanent_uuid: "c003e06bdebe4332828e1c10f60f3367" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33017 } }
I20260924 23:12:26.000316 24843 ts_tablet_manager.cc:1428] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:12:26.000420 24840 ts_tablet_manager.cc:1428] T 6f559286975f4ba5a3a238a851057d10 P 15cb3997f8a64c0cadb7e7c35830bd13: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:12:26.000480 24821 heartbeater.cc:503] Master 127.12.212.62:36605 was elected leader, sending a full tablet report...
I20260924 23:12:26.000557 24687 heartbeater.cc:503] Master 127.12.212.62:36605 was elected leader, sending a full tablet report...
Sep 24 23:12:26 dist-test-slave-m0z6 krb5kdc[23861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291545, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
I20260924 23:12:26.067040 24844 raft_consensus.cc:482] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:12:26.067186 24844 raft_consensus.cc:504] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15cb3997f8a64c0cadb7e7c35830bd13" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 43277 } } peers { permanent_uuid: "85c5899db90b43528987738c42309e3f" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46105 } } peers { permanent_uuid: "c003e06bdebe4332828e1c10f60f3367" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33017 } }
I20260924 23:12:26.067596 24844 leader_election.cc:288] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15cb3997f8a64c0cadb7e7c35830bd13 (127.12.212.2:43277), c003e06bdebe4332828e1c10f60f3367 (127.12.212.3:33017)
I20260924 23:12:26.072952 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:26.068450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:49629 (local address 127.12.212.2:43277)
0924 23:12:26.068563 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0924 23:12:26.068566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:12:26.069009 (+ 443us) server_negotiation.cc:407] Connection header received
0924 23:12:26.069120 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:12:26.069121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:12:26.069133 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:12:26.069161 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:12:26.069495 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:26.070258 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:26.072060 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:26.072550 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:26.072601 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:12:26.072611 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 23:12:26.072826 (+ 215us) server_negotiation.cc:299] Negotiation successful
0924 23:12:26.072872 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260924 23:12:26.074267 24849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:26.067871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:43277 (local address 127.12.212.1:49629)
0924 23:12:26.068918 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0924 23:12:26.068931 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:12:26.069043 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:12:26.069266 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:12:26.069266 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:12:26.069268 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:12:26.069356 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:12:26.069358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:12:26.070446 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:12:26.070448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:12:26.071934 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:12:26.071939 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:12:26.072666 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:12:26.072666 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:12:26.072735 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:12:26.072748 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 23:12:26.073332 (+ 584us) client_negotiation.cc:240] Negotiation successful
0924 23:12:26.073388 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1013,"thread_start_us":94,"threads_started":1}
I20260924 23:12:26.074819 24623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f559286975f4ba5a3a238a851057d10" candidate_uuid: "85c5899db90b43528987738c42309e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15cb3997f8a64c0cadb7e7c35830bd13" is_pre_election: true
I20260924 23:12:26.075074 24623 raft_consensus.cc:2454] T 6f559286975f4ba5a3a238a851057d10 P 15cb3997f8a64c0cadb7e7c35830bd13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85c5899db90b43528987738c42309e3f in term 0.
I20260924 23:12:26.075387 24436 leader_election.cc:302] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15cb3997f8a64c0cadb7e7c35830bd13, 85c5899db90b43528987738c42309e3f; no voters:
I20260924 23:12:26.075528 24844 raft_consensus.cc:2781] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:12:26.075567 24844 raft_consensus.cc:482] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:12:26.075585 24844 raft_consensus.cc:3037] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:12:26.076480 24844 raft_consensus.cc:504] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15cb3997f8a64c0cadb7e7c35830bd13" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 43277 } } peers { permanent_uuid: "85c5899db90b43528987738c42309e3f" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46105 } } peers { permanent_uuid: "c003e06bdebe4332828e1c10f60f3367" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33017 } }
I20260924 23:12:26.076632 24844 leader_election.cc:288] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f [CANDIDATE]: Term 1 election: Requested vote from peers 15cb3997f8a64c0cadb7e7c35830bd13 (127.12.212.2:43277), c003e06bdebe4332828e1c10f60f3367 (127.12.212.3:33017)
I20260924 23:12:26.077096 24623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f559286975f4ba5a3a238a851057d10" candidate_uuid: "85c5899db90b43528987738c42309e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15cb3997f8a64c0cadb7e7c35830bd13"
I20260924 23:12:26.077183 24623 raft_consensus.cc:3037] T 6f559286975f4ba5a3a238a851057d10 P 15cb3997f8a64c0cadb7e7c35830bd13 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:12:26.078049 24623 raft_consensus.cc:2454] T 6f559286975f4ba5a3a238a851057d10 P 15cb3997f8a64c0cadb7e7c35830bd13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85c5899db90b43528987738c42309e3f in term 1.
I20260924 23:12:26.078289 24436 leader_election.cc:302] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15cb3997f8a64c0cadb7e7c35830bd13, 85c5899db90b43528987738c42309e3f; no voters:
I20260924 23:12:26.078490 24844 raft_consensus.cc:2781] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:12:26.078648 24844 raft_consensus.cc:686] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f [term 1 LEADER]: Becoming Leader. State: Replica: 85c5899db90b43528987738c42309e3f, State: Running, Role: LEADER
I20260924 23:12:26.078770 24844 consensus_queue.cc:237] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15cb3997f8a64c0cadb7e7c35830bd13" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 43277 } } peers { permanent_uuid: "85c5899db90b43528987738c42309e3f" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46105 } } peers { permanent_uuid: "c003e06bdebe4332828e1c10f60f3367" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33017 } }
I20260924 23:12:26.079828 24337 catalog_manager.cc:5495] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f reported cstate change: term changed from 0 to 1, leader changed from <none> to 85c5899db90b43528987738c42309e3f (127.12.212.1). New cstate: current_term: 1 leader_uuid: "85c5899db90b43528987738c42309e3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15cb3997f8a64c0cadb7e7c35830bd13" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 43277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85c5899db90b43528987738c42309e3f" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c003e06bdebe4332828e1c10f60f3367" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260924 23:12:26.081594 24838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:26.068513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:55847 (local address 127.12.212.3:33017)
0924 23:12:26.068563 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 23:12:26.068566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:12:26.069014 (+ 448us) server_negotiation.cc:407] Connection header received
0924 23:12:26.069120 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:12:26.069121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:12:26.069130 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:12:26.069161 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:12:26.072704 (+ 3543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:26.073397 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:26.080360 (+ 6963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:26.081166 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:26.081226 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:12:26.081237 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0924 23:12:26.081477 (+ 240us) server_negotiation.cc:299] Negotiation successful
0924 23:12:26.081524 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 23:12:26.082023 24848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:26.068101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:33017 (local address 127.12.212.1:55847)
0924 23:12:26.068922 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0924 23:12:26.068931 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:12:26.069043 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:12:26.072524 (+ 3481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:12:26.072525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:12:26.072528 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:12:26.072601 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:12:26.072602 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:12:26.073529 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:12:26.073531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:12:26.080241 (+ 6710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:12:26.080247 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:12:26.081308 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:12:26.081310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:12:26.081396 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:12:26.081400 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:12:26.081857 (+ 457us) client_negotiation.cc:240] Negotiation successful
0924 23:12:26.081926 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"thread_start_us":71,"threads_started":1}
I20260924 23:12:26.082521 24756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f559286975f4ba5a3a238a851057d10" candidate_uuid: "85c5899db90b43528987738c42309e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c003e06bdebe4332828e1c10f60f3367" is_pre_election: true
I20260924 23:12:26.082676 24756 raft_consensus.cc:2454] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85c5899db90b43528987738c42309e3f in term 0.
I20260924 23:12:26.082808 24757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f559286975f4ba5a3a238a851057d10" candidate_uuid: "85c5899db90b43528987738c42309e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c003e06bdebe4332828e1c10f60f3367"
I20260924 23:12:26.082854 24757 raft_consensus.cc:3037] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:12:26.083734 24757 raft_consensus.cc:2454] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85c5899db90b43528987738c42309e3f in term 1.
I20260924 23:12:26.124014 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:26.120376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37686 (local address 127.12.212.1:46105)
0924 23:12:26.120438 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 23:12:26.120440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:12:26.120452 (+ 12us) server_negotiation.cc:407] Connection header received
0924 23:12:26.120553 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:12:26.120555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:12:26.120582 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:12:26.120663 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:12:26.120866 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:26.121667 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:26.122824 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:12:26.123082 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:12:26.123147 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:12:26.123367 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:12:26.123515 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:12:26.123627 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0924 23:12:26.123884 (+ 257us) server_negotiation.cc:299] Negotiation successful
0924 23:12:26.123908 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 23:12:26.126119 24623 raft_consensus.cc:1261] T 6f559286975f4ba5a3a238a851057d10 P 15cb3997f8a64c0cadb7e7c35830bd13 [term 1 FOLLOWER]: Refusing update from remote peer 85c5899db90b43528987738c42309e3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:12:26.126145 24757 raft_consensus.cc:1261] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367 [term 1 FOLLOWER]: Refusing update from remote peer 85c5899db90b43528987738c42309e3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:12:26.126515 24850 consensus_queue.cc:1035] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c003e06bdebe4332828e1c10f60f3367" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:12:26.126713 24850 consensus_queue.cc:1035] T 6f559286975f4ba5a3a238a851057d10 P 85c5899db90b43528987738c42309e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "15cb3997f8a64c0cadb7e7c35830bd13" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 43277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:12:26.137625 24853 mvcc.cc:205] Tried to move back new op lower bound from 7333034172929601536 to 7333034172739260416. Current Snapshot: MvccSnapshot[applied={T|T < 7333034172929601536}]
I20260924 23:12:26.139106 24856 mvcc.cc:205] Tried to move back new op lower bound from 7333034172929601536 to 7333034172739260416. Current Snapshot: MvccSnapshot[applied={T|T < 7333034172929601536}]
I20260924 23:12:26.142378 24337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:12:26.142493 24337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:12:26.143518 24337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a0de329b221847849db3051fd5971ae6: Sending DeleteTablet for 3 replicas of tablet 6f559286975f4ba5a3a238a851057d10
I20260924 23:12:26.144050 24618 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f559286975f4ba5a3a238a851057d10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:12:26 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:41204
I20260924 23:12:26.144307 24744 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f559286975f4ba5a3a238a851057d10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:12:26 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:48362
I20260924 23:12:26.144776 24465 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f559286975f4ba5a3a238a851057d10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:12:26 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:37676
I20260924 23:12:26.145493 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 24418
I20260924 23:12:26.149725 24862 tablet_replica.cc:330] T 6f559286975f4ba5a3a238a851057d10 P 15cb3997f8a64c0cadb7e7c35830bd13: stopping tablet replica
I20260924 23:12:26.149951 24862 raft_consensus.cc:2229] T 6f559286975f4ba5a3a238a851057d10 P 15cb3997f8a64c0cadb7e7c35830bd13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:12:26.150168 24863 tablet_replica.cc:330] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367: stopping tablet replica
I20260924 23:12:26.150313 24862 raft_consensus.cc:2258] T 6f559286975f4ba5a3a238a851057d10 P 15cb3997f8a64c0cadb7e7c35830bd13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:12:26.150460 24863 raft_consensus.cc:2229] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:12:26.150686 24863 raft_consensus.cc:2258] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:12:26.154951 24863 ts_tablet_manager.cc:1905] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:12:26.157397 24863 ts_tablet_manager.cc:1918] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:12:26.158141 24862 ts_tablet_manager.cc:1905] T 6f559286975f4ba5a3a238a851057d10 P 15cb3997f8a64c0cadb7e7c35830bd13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:12:26.158946 24863 log.cc:1196] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/6f559286975f4ba5a3a238a851057d10
I20260924 23:12:26.159281 24863 ts_tablet_manager.cc:1939] T 6f559286975f4ba5a3a238a851057d10 P c003e06bdebe4332828e1c10f60f3367: Deleting consensus metadata
I20260924 23:12:26.159970 24321 catalog_manager.cc:4841] TS c003e06bdebe4332828e1c10f60f3367 (127.12.212.3:33017): tablet 6f559286975f4ba5a3a238a851057d10 (table test-table [id=d10789cf0f354ab8ac71ceef28937188]) successfully deleted
I20260924 23:12:26.160112 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 24555
W20260924 23:12:26.160581 24322 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.212.1:46105 (error 108)
I20260924 23:12:26.161346 24833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:26.161230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:46105 (local address 127.0.0.1:37690)
0924 23:12:26.161280 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 23:12:26.161313 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.1:46105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 23:12:26.161449 24322 catalog_manager.cc:4584] TS 85c5899db90b43528987738c42309e3f (127.12.212.1:46105): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f559286975f4ba5a3a238a851057d10: Network error: Client connection negotiation failed: client connection to 127.12.212.1:46105: connect: Connection refused (error 111)
I20260924 23:12:26.165426 24833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:12:26.165300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:43277 (local address 127.0.0.1:41214)
0924 23:12:26.165355 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0924 23:12:26.165382 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.2:43277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260924 23:12:26.165648 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 24692
I20260924 23:12:26.171684 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 24305
I20260924 23:12:26.178006 13136 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260924 23:12:26.513266 13136 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260924 23:12:26.513373 13136 mini_ranger.cc:66] Stopping Ranger...
I20260924 23:12:26.848414 13136 mini_ranger.cc:68] Stopped Ranger
2026-09-24 23:12:26.848 UTC [23891] LOG: received smart shutdown request
2026-09-24 23:12:26.851 UTC [23891] LOG: background worker "logical replication launcher" (PID 23900) exited with exit code 1
2026-09-24 23:12:26.851 UTC [23895] LOG: shutting down
2026-09-24 23:12:27.052 UTC [23891] LOG: database system is shut down
2026-09-24T23:12:27Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48410 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:12:27 dist-test-slave-m0z6 krb5kdc[24874](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:12:27 dist-test-slave-m0z6 krb5kdc[24874](info): set up 2 sockets
Sep 24 23:12:27 dist-test-slave-m0z6 krb5kdc[24874](info): commencing operation
krb5kdc: starting...
W20260924 23:12:28.114459 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.953s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:12:28 dist-test-slave-m0z6 krb5kdc[24874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:12:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:12:28Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.12.212.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.12.212.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.12.212.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/rangeradmin_127.12.212.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.12.212.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/rangeradmin_127.12.212.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.12.212.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.12.212.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.12.212.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/rangerlookup_127.12.212.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.12.212.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/rangerlookup_127.12.212.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.212.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.12.212.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.212.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/HTTP_127.12.212.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.212.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/HTTP_127.12.212.20@KRBTEST.COM.keytab.
I20260924 23:12:28.279209 13136 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskmL_OPf/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/postgres -l logfile start
2026-09-24 23:12:30.955 UTC [24904] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-24 23:12:30.955 UTC [24904] LOG: listening on IPv4 address "127.12.212.20", port 48647
2026-09-24 23:12:30.958 UTC [24904] LOG: listening on Unix socket "/tmp/.s.PGSQL.48647"
2026-09-24 23:12:30.988 UTC [24907] LOG: database system was shut down at 2026-09-24 23:12:29 UTC
2026-09-24 23:12:30.992 UTC [24904] LOG: database system is ready to accept connections
I20260924 23:12:31.862437 13136 mini_postgres.cc:96] Postgres bound to 48647
2026-09-24 23:12:31.868 UTC [24915] FATAL: database "slave" does not exist
127.12.212.20:48647 - accepting connections
I20260924 23:12:31.868884 13136 mini_ranger.cc:161] Starting Ranger...
I20260924 23:12:31.883831 13136 mini_ranger.cc:84] Created miniranger Postgres user
I20260924 23:12:32.458593 13136 mini_ranger.cc:87] Created ranger Postgres database
I20260924 23:12:32.458709 13136 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ranger-admin
2026-09-24 23:12:32,736 [I] DB FLAVOR :POSTGRES
2026-09-24 23:12:32,736 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 23:12:32,736 [I] Checking connection..
2026-09-24 23:12:32,737 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-24 23:12:32,989 [I] Checking connection passed.
2026-09-24 23:12:32,990 [I] --------- Verifying version history table ---------
2026-09-24 23:12:32,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 23:12:33,253 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 23:12:33,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 23:12:33,499 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 23:12:33,499 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-24 23:12:33,499 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-24 23:12:33.770 UTC [24979] WARNING: there is no transaction in progress
2026-09-24 23:12:33,776 [I] create_dbversion_catalog.sql file imported successfully
2026-09-24 23:12:33,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 23:12:34,040 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-24 23:12:34,040 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 23:12:34,041 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-24 23:12:34,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-24 23:12:34,550 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-m0z6.c.gcp-upstream.internal','N') ;"
2026-09-24 23:12:34,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 23:12:35,058 [I] Table x_portal_user does not exist in database ranger
2026-09-24 23:12:35,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 23:12:35,332 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-24 23:12:35,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 23:12:35,600 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 23:12:35,853 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-24 23:12:35,853 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-24 23:12:36.972 UTC [25087] WARNING: there is no transaction in progress
2026-09-24 23:12:36.992 UTC [25087] WARNING: there is no transaction in progress
2026-09-24 23:12:37.011 UTC [25087] WARNING: there is no transaction in progress
2026-09-24 23:12:37.031 UTC [25087] WARNING: there is no transaction in progress
2026-09-24 23:12:37.053 UTC [25087] WARNING: there is no transaction in progress
2026-09-24 23:12:37.074 UTC [25087] WARNING: there is no transaction in progress
2026-09-24 23:12:37.139 UTC [25087] WARNING: there is no transaction in progress
2026-09-24 23:12:37.146 UTC [25087] WARNING: there is no transaction in progress
2026-09-24 23:12:37.162 UTC [25087] WARNING: there is no transaction in progress
2026-09-24 23:12:37.175 UTC [25087] WARNING: there is no transaction in progress
2026-09-24 23:12:37.187 UTC [25087] WARNING: there is no transaction in progress
2026-09-24 23:12:37.202 UTC [25087] WARNING: there is no transaction in progress
2026-09-24 23:12:37.212 UTC [25087] WARNING: there is no transaction in progress
2026-09-24 23:12:37.220 UTC [25087] WARNING: there is no transaction in progress
2026-09-24 23:12:37.228 UTC [25087] WARNING: there is no transaction in progress
2026-09-24 23:12:37.236 UTC [25087] WARNING: there is no transaction in progress
2026-09-24 23:12:38,024 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-24 23:12:38,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-24 23:12:38,279 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-24 23:12:38,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 23:12:38,517 [I] Table x_portal_user already exists in database 'ranger'
2026-09-24 23:12:38,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 23:12:38,760 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-24 23:12:38,760 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 23:12:39,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 23:12:39,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-m0z6.c.gcp-upstream.internal';"
2026-09-24 23:12:39,522 [I] CORE_DB_SCHEMA import status has been updated
2026-09-24 23:12:39,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-24 23:12:39,773 [I] DB_PATCHES have already been applied
I20260924 23:12:39.779206 13136 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ranger-admin:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskmL_OPf/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260924 23:12:39.779275 13136 mini_ranger.cc:193] Using host: 127.12.212.20
I20260924 23:12:39.782377 13136 mini_ranger.cc:227] Ranger admin URL: http://127.12.212.20:36887
Sep 24, 2026 11:12:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 11:12:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 11:12:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ranger-admin/ews/webapp
Sep 24, 2026 11:12:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 24, 2026 11:12:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 24, 2026 11:12:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ranger-admin/ews/webapp].
Sep 24, 2026 11:12:40 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260924 23:12:40.812301 13136 mini_ranger.cc:160] Time spent starting Ranger: real 8.943s user 0.002s sys 0.006s
Sep 24, 2026 11:13:02 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 24, 2026 11:13:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 24, 2026 11:13:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 24, 2026 11:13:02 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 24, 2026 11:13:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 24, 2026 11:13:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 24, 2026 11:13:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 24, 2026 11:13:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 24, 2026 11:13:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 24, 2026 11:13:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 24, 2026 11:13:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 24, 2026 11:13:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 24, 2026 11:13:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 24, 2026 11:13:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 24, 2026 11:13:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 24, 2026 11:13:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 24, 2026 11:13:02 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 24, 2026 11:13:03 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260924 23:13:03.870630 13136 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.12.212.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.12.212.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.12.212.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/rangerkms_127.12.212.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.12.212.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/rangerkms_127.12.212.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.212.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.12.212.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.212.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/HTTP_127.12.212.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.212.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/HTTP_127.12.212.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 24 23:13:03 dist-test-slave-m0z6 krb5kdc[24874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291583, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260924 23:13:03.953516 13136 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260924 23:13:03.963337 13136 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260924 23:13:04.888924 13136 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260924 23:13:04.888999 13136 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ranger-kms
I20260924 23:13:04.889011 13136 mini_ranger_kms.cc:227] Using postgres at 127.12.212.20:48647
2026-09-24 23:13:05,041 [I] DB FLAVOR :POSTGRES
2026-09-24 23:13:05,042 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 23:13:05,042 [I] Checking connection
2026-09-24 23:13:05,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-24 23:13:05,284 [I] connection success
2026-09-24 23:13:05,284 [I] --------- Verifying Ranger DB tables ---------
2026-09-24 23:13:05,285 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-24 23:13:05,285 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-24 23:13:05,520 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-24 23:13:05,521 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 23:13:05,521 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-24 23:13:05,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.212.20:48647/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-24 23:13:05,786 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260924 23:13:06.321187 13136 mini_ranger_kms.cc:310] Created kms service
I20260924 23:13:06.527546 13136 mini_ranger_kms.cc:326] Created kudu user
I20260924 23:13:06.577905 13136 mini_ranger_kms.cc:343] Created rangerkms user
Sep 24 23:13:06 dist-test-slave-m0z6 krb5kdc[24874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291583, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
I20260924 23:13:07.026583 13136 mini_ranger_kms.cc:384] Added ranger policy
I20260924 23:13:07.026840 13136 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ranger-kms:/tmp/dist-test-taskmL_OPf/build/release/bin/postgresql.jar:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskmL_OPf/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260924 23:13:07.026865 13136 mini_ranger_kms.cc:240] Using host: 127.12.212.20
I20260924 23:13:07.030452 13136 mini_ranger_kms.cc:276] Ranger KMS PID: 25268
I20260924 23:13:07.030541 13136 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.12.212.20:38651
Sep 24, 2026 11:13:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 11:13:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 11:13:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 24, 2026 11:13:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 24, 2026 11:13:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ranger-kms/ews/webapp].
Sep 24, 2026 11:13:08 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260924 23:13:09.072269 13136 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.119s user 0.007s sys 0.128s
I20260924 23:13:09.072356 13136 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260924 23:13:09.072391 13136 mini_ranger_kms.cc:401] 127.12.212.20:38651/kms/v1/keys
Sep 24 23:13:09 dist-test-slave-m0z6 krb5kdc[24874](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790291589, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.212.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 24 23:13:09 dist-test-slave-m0z6 krb5kdc[24874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291589, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.212.20@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
Sep 24 23:13:10 dist-test-slave-m0z6 krb5kdc[24874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291589, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.212.20@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
Sep 24 23:13:12 dist-test-slave-m0z6 krb5kdc[24874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:13:12.641934 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:43659
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:43659
--ranger_config_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.212.20:38651/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:13:12.723953 25318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:12.724192 25318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:12.724216 25318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:12.725548 25318 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:13:12.725610 25318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:12.725625 25318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:12.725636 25318 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:13:12.725646 25318 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:13:12.727440 25318 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/wal
--ranger_kms_url=127.12.212.20:38651/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:43659
--ranger_config_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:43659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:12.727676 25318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:12.727859 25318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:12.733037 25323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:12.733026 25326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:12.733026 25324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:12.733166 25318 server_base.cc:1034] running on GCE node
I20260924 23:13:12.733664 25318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:12.733989 25318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:12.735169 25318 hybrid_clock.cc:648] HybridClock initialized: now 1790291592735145 us; error 44 us; skew 500 ppm
Sep 24 23:13:12 dist-test-slave-m0z6 krb5kdc[24874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291592, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:12.742602 25318 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:13:12.743165 25318 webserver.cc:460] Webserver started at http://127.12.212.62:36387/ using document root <none> and password file <none>
I20260924 23:13:12.743394 25318 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:12.743436 25318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:12.743587 25318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 23:13:12 dist-test-slave-m0z6 krb5kdc[24874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291592, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
I20260924 23:13:12.809770 25318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "fad188054df44188baa7435fa618e3be"
format_stamp: "Formatted at 2026-09-24 23:13:12 on dist-test-slave-m0z6"
server_key: "99a8d64ddd94239ebc91e310407c888d"
server_key_iv: "08fa38629efd23beb35d29024fe39d30"
server_key_version: "kuduclusterkey@0"
I20260924 23:13:12.810132 25318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "fad188054df44188baa7435fa618e3be"
format_stamp: "Formatted at 2026-09-24 23:13:12 on dist-test-slave-m0z6"
server_key: "99a8d64ddd94239ebc91e310407c888d"
server_key_iv: "08fa38629efd23beb35d29024fe39d30"
server_key_version: "kuduclusterkey@0"
I20260924 23:13:12.811612 25318 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:13:12.828006 25333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:12.828274 25318 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 23:13:12.828378 25318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "fad188054df44188baa7435fa618e3be"
format_stamp: "Formatted at 2026-09-24 23:13:12 on dist-test-slave-m0z6"
server_key: "99a8d64ddd94239ebc91e310407c888d"
server_key_iv: "08fa38629efd23beb35d29024fe39d30"
server_key_version: "kuduclusterkey@0"
I20260924 23:13:12.828480 25318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:12.852591 25318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:12.854669 25318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:12.854867 25318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:12.859352 25318 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:43659
I20260924 23:13:12.859508 25385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:43659 every 8 connection(s)
I20260924 23:13:12.859804 25318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:13:12.860546 25386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:12.862970 25386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be: Bootstrap starting.
I20260924 23:13:12.863579 25386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be: Neither blocks nor log segments found. Creating new log.
I20260924 23:13:12.863860 25386 log.cc:824] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:12.864689 25386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be: No bootstrap required, opened a new log
I20260924 23:13:12.866880 25386 raft_consensus.cc:348] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fad188054df44188baa7435fa618e3be" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 43659 } }
I20260924 23:13:12.867050 25386 raft_consensus.cc:374] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:13:12.867072 25386 raft_consensus.cc:729] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fad188054df44188baa7435fa618e3be, State: Initialized, Role: FOLLOWER
I20260924 23:13:12.867272 25386 consensus_queue.cc:260] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fad188054df44188baa7435fa618e3be" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 43659 } }
I20260924 23:13:12.867403 25386 raft_consensus.cc:388] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:13:12.867462 25386 raft_consensus.cc:482] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:13:12.867522 25386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:13:12.868206 25386 raft_consensus.cc:504] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fad188054df44188baa7435fa618e3be" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 43659 } }
I20260924 23:13:12.868315 25386 leader_election.cc:302] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fad188054df44188baa7435fa618e3be; no voters:
I20260924 23:13:12.868484 25386 leader_election.cc:288] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:13:12.868562 25389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:13:12.868765 25389 raft_consensus.cc:686] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [term 1 LEADER]: Becoming Leader. State: Replica: fad188054df44188baa7435fa618e3be, State: Running, Role: LEADER
I20260924 23:13:12.868774 25386 sys_catalog.cc:564] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:13:12.868983 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 25318
I20260924 23:13:12.869094 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:13:12.869050 25389 consensus_queue.cc:237] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fad188054df44188baa7435fa618e3be" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 43659 } }
I20260924 23:13:12.869303 13136 external_mini_cluster.cc:1428] Setting key b382fc67f7be09b496bbc93a6a56a2a7
I20260924 23:13:12.869529 25391 sys_catalog.cc:455] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [sys.catalog]: SysCatalogTable state changed. Reason: New leader fad188054df44188baa7435fa618e3be. Latest consensus state: current_term: 1 leader_uuid: "fad188054df44188baa7435fa618e3be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fad188054df44188baa7435fa618e3be" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 43659 } } }
I20260924 23:13:12.869616 25391 sys_catalog.cc:458] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [sys.catalog]: This master's current role is: LEADER
I20260924 23:13:12.869848 25390 sys_catalog.cc:455] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fad188054df44188baa7435fa618e3be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fad188054df44188baa7435fa618e3be" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 43659 } } }
I20260924 23:13:12.869936 25390 sys_catalog.cc:458] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [sys.catalog]: This master's current role is: LEADER
I20260924 23:13:12.870438 25398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:13:12.870661 25386 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 23:13:12.871025 25398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 23:13:12 dist-test-slave-m0z6 krb5kdc[24874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:12.879312 25398 catalog_manager.cc:1347] Generated new cluster ID: f76bff98391541dfbe2a6273a3213a4e
I20260924 23:13:12.879377 25398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:13:12.884126 25401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:12.869970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47850 (local address 127.12.212.62:43659)
0924 23:13:12.870382 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:12.870393 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:12.870497 (+ 104us) server_negotiation.cc:407] Connection header received
0924 23:13:12.870656 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:12.870664 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:12.870755 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:12.870907 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:12.872598 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:12.874736 (+ 2138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:12.875883 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:12.876503 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:12.876636 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:12.879714 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:12.879719 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:12.879719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:12.879733 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:12.881830 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:12.882287 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:12.882290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:12.882291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:12.882402 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:12.882624 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:12.882625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:12.882625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:12.883011 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:12.883676 (+ 665us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:12.883754 (+ 78us) server_negotiation.cc:299] Negotiation successful
0924 23:13:12.883889 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":83,"threads_started":1}
I20260924 23:13:12.926728 25398 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:13:12.927510 25398 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:13:12.937637 25398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be: Generated new TSK 0
I20260924 23:13:12.937904 25398 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
I20260924 23:13:13.653930 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.212.20:38651/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:43659
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:13:13.743094 25431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:13.743328 25431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:13.743361 25431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:13.744656 25431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:13.744710 25431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:13.744751 25431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:13:13.746528 25431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.12.212.20:38651/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:43659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:13.746820 25431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:13.747068 25431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:13.752019 25439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:13.752019 25437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:13.752332 25436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:13.752772 25431 server_base.cc:1034] running on GCE node
I20260924 23:13:13.753000 25431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:13.753240 25431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:13.756543 25431 hybrid_clock.cc:648] HybridClock initialized: now 1790291593756521 us; error 35 us; skew 500 ppm
Sep 24 23:13:13 dist-test-slave-m0z6 krb5kdc[24874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291593, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:13.759145 25431 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:13:13.759613 25431 webserver.cc:460] Webserver started at http://127.12.212.1:34791/ using document root <none> and password file <none>
I20260924 23:13:13.759828 25431 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:13.759879 25431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:13.760003 25431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 23:13:13 dist-test-slave-m0z6 krb5kdc[24874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291593, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
I20260924 23:13:13.774652 25431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "074eec832c8748c98fcdb51fbc80522e"
format_stamp: "Formatted at 2026-09-24 23:13:13 on dist-test-slave-m0z6"
server_key: "0eef6355b33f065413bb8347e2f63caf"
server_key_iv: "3824a6875775063847915dec9041eba6"
server_key_version: "kuduclusterkey@0"
I20260924 23:13:13.775214 25431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "074eec832c8748c98fcdb51fbc80522e"
format_stamp: "Formatted at 2026-09-24 23:13:13 on dist-test-slave-m0z6"
server_key: "0eef6355b33f065413bb8347e2f63caf"
server_key_iv: "3824a6875775063847915dec9041eba6"
server_key_version: "kuduclusterkey@0"
I20260924 23:13:13.777400 25431 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260924 23:13:13.792258 25446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:13.792506 25431 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:13.792606 25431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "074eec832c8748c98fcdb51fbc80522e"
format_stamp: "Formatted at 2026-09-24 23:13:13 on dist-test-slave-m0z6"
server_key: "0eef6355b33f065413bb8347e2f63caf"
server_key_iv: "3824a6875775063847915dec9041eba6"
server_key_version: "kuduclusterkey@0"
I20260924 23:13:13.792692 25431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:13.803674 25431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:13.806008 25431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:13.806174 25431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:13.806394 25431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:13:13.806784 25431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:13:13.806824 25431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:13.806854 25431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:13:13.806892 25431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:13.813354 25431 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:42907
I20260924 23:13:13.813722 25431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:13:13.814209 25559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:42907 every 8 connection(s)
I20260924 23:13:13.819453 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 25431
I20260924 23:13:13.819558 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:13:13.819746 13136 external_mini_cluster.cc:1428] Setting key 24c5497f99152c7e3991a96dc8dc1685
Sep 24 23:13:13 dist-test-slave-m0z6 krb5kdc[24874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291593, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:13.827211 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:13.814791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:41065 (local address 127.12.212.62:43659)
0924 23:13:13.818826 (+ 4035us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:13.818828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:13.818837 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:13:13.818861 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:13.818861 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:13.818873 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:13.818937 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:13.819666 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:13.820437 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:13.821933 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:13.822218 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:13.822278 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:13.824504 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:13.824506 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:13.824506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:13.824514 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:13.825896 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:13.826274 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:13.826275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:13.826275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:13.826336 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:13.826551 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:13.826551 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:13.826552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:13.826667 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:13.826763 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:13.827024 (+ 261us) server_negotiation.cc:299] Negotiation successful
0924 23:13:13.827110 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4004,"thread_start_us":84,"threads_started":1}
I20260924 23:13:13.827381 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:13.814414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:43659 (local address 127.12.212.1:41065)
0924 23:13:13.814559 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:13.814601 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:13.815154 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:13.819044 (+ 3890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:13.819048 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:13.819234 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:13.819546 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:13.819554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:13.820542 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:13.820543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:13.821824 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:13.821829 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:13.822366 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:13.822367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:13.822510 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:13.823131 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:13.823142 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:13.824413 (+ 1271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:13.825970 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:13.825976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:13.825978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:13.826208 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:13.826390 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:13.826391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:13.826391 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:13.826497 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:13.826757 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:13.826763 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:13.826906 (+ 143us) client_negotiation.cc:769] Sending connection context
0924 23:13:13.826998 (+ 92us) client_negotiation.cc:240] Negotiation successful
0924 23:13:13.827148 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":81,"threads_started":1}
I20260924 23:13:13.828787 25560 heartbeater.cc:347] Connected to a master server at 127.12.212.62:43659
I20260924 23:13:13.828888 25560 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:13.829087 25560 heartbeater.cc:511] Master 127.12.212.62:43659 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.12.212.2@KRBTEST.COM; defaulting to no policy
I20260924 23:13:13.829597 25350 ts_manager.cc:194] Registered new tserver with Master: 074eec832c8748c98fcdb51fbc80522e (127.12.212.1:42907)
I20260924 23:13:13.830336 25350 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:41065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
I20260924 23:13:13.849620 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.212.20:38651/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:43659
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:13:13.968591 25568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:13.968832 25568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:13.968865 25568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:13.970175 25568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:13.970222 25568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:13.970263 25568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:13:13.971896 25568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.12.212.20:38651/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:43659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:13.972160 25568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:13.972465 25568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:13.977362 25576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:13.977504 25578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:13.977362 25575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:13.978076 25568 server_base.cc:1034] running on GCE node
I20260924 23:13:13.978312 25568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:13.978562 25568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:13.985970 25568 hybrid_clock.cc:648] HybridClock initialized: now 1790291593985929 us; error 39 us; skew 500 ppm
Sep 24 23:13:13 dist-test-slave-m0z6 krb5kdc[24874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291593, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:13.988816 25568 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:13:13.989408 25568 webserver.cc:460] Webserver started at http://127.12.212.2:36013/ using document root <none> and password file <none>
I20260924 23:13:13.989652 25568 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:13.989704 25568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:13.989861 25568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 23:13:13 dist-test-slave-m0z6 krb5kdc[24874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291593, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
I20260924 23:13:14.005882 25568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "aca7815dbeb54667ae2af17dfaeb500b"
format_stamp: "Formatted at 2026-09-24 23:13:13 on dist-test-slave-m0z6"
server_key: "b3c9bc7a0cb5607187b19fee59f4f9ea"
server_key_iv: "52143bec96fb6018d20c620575b9b81e"
server_key_version: "kuduclusterkey@0"
I20260924 23:13:14.006353 25568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "aca7815dbeb54667ae2af17dfaeb500b"
format_stamp: "Formatted at 2026-09-24 23:13:13 on dist-test-slave-m0z6"
server_key: "b3c9bc7a0cb5607187b19fee59f4f9ea"
server_key_iv: "52143bec96fb6018d20c620575b9b81e"
server_key_version: "kuduclusterkey@0"
I20260924 23:13:14.008426 25568 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260924 23:13:14.024375 25585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:14.024542 25568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 23:13:14.024629 25568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "aca7815dbeb54667ae2af17dfaeb500b"
format_stamp: "Formatted at 2026-09-24 23:13:13 on dist-test-slave-m0z6"
server_key: "b3c9bc7a0cb5607187b19fee59f4f9ea"
server_key_iv: "52143bec96fb6018d20c620575b9b81e"
server_key_version: "kuduclusterkey@0"
I20260924 23:13:14.024717 25568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Sep 24 23:13:14 dist-test-slave-m0z6 krb5kdc[24874](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790291594, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:14.043732 25568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:14.046064 25568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:14.046298 25568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:14.046589 25568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:13:14.047010 25568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:13:14.047077 25568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:14.047127 25568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:13:14.047173 25568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:14.053352 25568 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:41629
I20260924 23:13:14.053920 25568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:13:14.055224 25698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:41629 every 8 connection(s)
I20260924 23:13:14.061055 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 25568
I20260924 23:13:14.061182 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:13:14.061362 13136 external_mini_cluster.cc:1428] Setting key 99e39650269f4a5bad9bb5c473ded3c0
Sep 24 23:13:14 dist-test-slave-m0z6 krb5kdc[24874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291593, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:14.066478 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:14.056293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:35461 (local address 127.12.212.62:43659)
0924 23:13:14.056359 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:14.056361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:14.056598 (+ 237us) server_negotiation.cc:407] Connection header received
0924 23:13:14.057334 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:14.057336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:14.057353 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:14.057386 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:14.058276 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:14.059120 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:14.060635 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:14.060876 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:14.060937 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:14.063506 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:14.063507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:14.063508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:14.063517 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:14.065027 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:14.065400 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:14.065401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:14.065402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:14.065462 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:14.065727 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:14.065728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:14.065728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:14.065861 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:14.065900 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:14.066271 (+ 371us) server_negotiation.cc:299] Negotiation successful
0924 23:13:14.066365 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 23:13:14.067065 25701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:14.056301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:43659 (local address 127.12.212.2:35461)
0924 23:13:14.056468 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:14.056513 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:14.057196 (+ 683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:14.057590 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:14.057595 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:14.057852 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:14.058143 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:14.058162 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:14.059246 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:14.059247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:14.060541 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:14.060544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:14.061021 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:14.061023 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:14.061165 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:14.061765 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:14.061775 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:14.063389 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:14.065105 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:14.065110 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:14.065112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:14.065321 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:14.065526 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:14.065527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:14.065527 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:14.065660 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:14.065939 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:14.065944 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:14.066113 (+ 169us) client_negotiation.cc:769] Sending connection context
0924 23:13:14.066616 (+ 503us) client_negotiation.cc:240] Negotiation successful
0924 23:13:14.066821 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260924 23:13:14.067653 25699 heartbeater.cc:347] Connected to a master server at 127.12.212.62:43659
I20260924 23:13:14.067754 25699 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:14.067945 25699 heartbeater.cc:511] Master 127.12.212.62:43659 requested a full tablet report, sending...
I20260924 23:13:14.068379 25350 ts_manager.cc:194] Registered new tserver with Master: aca7815dbeb54667ae2af17dfaeb500b (127.12.212.2:41629)
I20260924 23:13:14.068862 25350 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:35461
WARNING: no policy specified for kudu/127.12.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
I20260924 23:13:14.093520 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.212.20:38651/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:43659
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:13:14.183378 25704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:14.183604 25704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:14.183637 25704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:14.184994 25704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:14.185055 25704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:14.185092 25704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:13:14.187031 25704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.12.212.20:38651/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:43659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:14.187460 25704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:14.187776 25704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:14.193306 25713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:14.193547 25710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:14.193598 25711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:14.193877 25704 server_base.cc:1034] running on GCE node
I20260924 23:13:14.194083 25704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:14.194316 25704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:14.195487 25704 hybrid_clock.cc:648] HybridClock initialized: now 1790291594195477 us; error 31 us; skew 500 ppm
Sep 24 23:13:14 dist-test-slave-m0z6 krb5kdc[24874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291594, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:14.198346 25704 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:13:14.198906 25704 webserver.cc:460] Webserver started at http://127.12.212.3:39775/ using document root <none> and password file <none>
I20260924 23:13:14.199124 25704 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:14.199173 25704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:14.199314 25704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 23:13:14 dist-test-slave-m0z6 krb5kdc[24874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291594, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
I20260924 23:13:14.213640 25704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "36ce009abb1b4461b3afacacd6965959"
format_stamp: "Formatted at 2026-09-24 23:13:14 on dist-test-slave-m0z6"
server_key: "57144c06fbb5cfcb162993dd53ba45bb"
server_key_iv: "d0a5eb6f099b8bfa0fbe5e12a2537755"
server_key_version: "kuduclusterkey@0"
I20260924 23:13:14.214218 25704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "36ce009abb1b4461b3afacacd6965959"
format_stamp: "Formatted at 2026-09-24 23:13:14 on dist-test-slave-m0z6"
server_key: "57144c06fbb5cfcb162993dd53ba45bb"
server_key_iv: "d0a5eb6f099b8bfa0fbe5e12a2537755"
server_key_version: "kuduclusterkey@0"
I20260924 23:13:14.223232 25704 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.001s
I20260924 23:13:14.245184 25720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:14.246013 25704 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260924 23:13:14.246119 25704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "36ce009abb1b4461b3afacacd6965959"
format_stamp: "Formatted at 2026-09-24 23:13:14 on dist-test-slave-m0z6"
server_key: "57144c06fbb5cfcb162993dd53ba45bb"
server_key_iv: "d0a5eb6f099b8bfa0fbe5e12a2537755"
server_key_version: "kuduclusterkey@0"
I20260924 23:13:14.246210 25704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:14.257184 25704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:14.259212 25704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:14.259359 25704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:14.259569 25704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:13:14.259944 25704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:13:14.259984 25704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:14.260015 25704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:13:14.260066 25704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:14.267376 25704 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:36249
I20260924 23:13:14.267989 25704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:13:14.269979 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 25704
I20260924 23:13:14.270088 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:13:14.270275 13136 external_mini_cluster.cc:1428] Setting key 7d3e662cd19fe5e13c03b9f779906f91
I20260924 23:13:14.272656 25833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:36249 every 8 connection(s)
Sep 24 23:13:14 dist-test-slave-m0z6 krb5kdc[24874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291594, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:14.284198 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:14.273636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:48389 (local address 127.12.212.62:43659)
0924 23:13:14.273692 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:14.273694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:14.274067 (+ 373us) server_negotiation.cc:407] Connection header received
0924 23:13:14.274797 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:14.274798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:14.274811 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:14.274842 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:14.275730 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:14.276452 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:14.277622 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:14.277938 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:14.277985 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:14.280795 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:14.280797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:14.280798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:14.280806 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:14.282494 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:14.283033 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:14.283034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:14.283034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:14.283103 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:14.283409 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:14.283410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:14.283410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:14.283556 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:14.283610 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:14.284011 (+ 401us) server_negotiation.cc:299] Negotiation successful
0924 23:13:14.284100 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 23:13:14.285010 25836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:14.273350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:43659 (local address 127.12.212.3:48389)
0924 23:13:14.273939 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:14.273983 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:14.274636 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:14.275049 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:14.275054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:14.275308 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:14.275604 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:14.275612 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:14.276544 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:14.276545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:14.277524 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:14.277527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:14.278095 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:14.278096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:14.278242 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:14.278910 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:14.278921 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:14.280658 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:14.282591 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:14.282598 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:14.282601 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:14.282943 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:14.283175 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:14.283175 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:14.283176 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:14.283326 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:14.283645 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:14.283651 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:14.283850 (+ 199us) client_negotiation.cc:769] Sending connection context
0924 23:13:14.284568 (+ 718us) client_negotiation.cc:240] Negotiation successful
0924 23:13:14.284763 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":88,"threads_started":1}
I20260924 23:13:14.285768 25834 heartbeater.cc:347] Connected to a master server at 127.12.212.62:43659
I20260924 23:13:14.285866 25834 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:14.286041 25834 heartbeater.cc:511] Master 127.12.212.62:43659 requested a full tablet report, sending...
I20260924 23:13:14.286396 25350 ts_manager.cc:194] Registered new tserver with Master: 36ce009abb1b4461b3afacacd6965959 (127.12.212.3:36249)
I20260924 23:13:14.286816 25350 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:48389
I20260924 23:13:14.293095 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 23:13:14.293653 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 25431
I20260924 23:13:14.298066 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 25568
I20260924 23:13:14.302559 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 25704
I20260924 23:13:14.307905 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 25318
I20260924 23:13:14.312916 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:43659
--webserver_interface=127.12.212.62
--webserver_port=36387
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:43659
--ranger_config_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.212.20:38651/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.212.20:38651/kms with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:13:14.393759 25837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:14.393996 25837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:14.394018 25837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:14.395220 25837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:13:14.395265 25837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:14.395278 25837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:14.395291 25837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:13:14.395303 25837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:13:14.396890 25837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.212.20:38651/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:43659
--ranger_config_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:43659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=36387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:14.397092 25837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:14.397271 25837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:13:14.402392 25837 server_base.cc:1034] running on GCE node
W20260924 23:13:14.402338 25842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:14.402330 25845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:14.402330 25843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:14.402928 25837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:14.403232 25837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:14.404412 25837 hybrid_clock.cc:648] HybridClock initialized: now 1790291594404396 us; error 35 us; skew 500 ppm
Sep 24 23:13:14 dist-test-slave-m0z6 krb5kdc[24874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291594, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:14.406965 25837 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:13:14.407402 25837 webserver.cc:460] Webserver started at http://127.12.212.62:36387/ using document root <none> and password file <none>
I20260924 23:13:14.407600 25837 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:14.407652 25837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 23:13:14 dist-test-slave-m0z6 krb5kdc[24874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291594, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
I20260924 23:13:14.452030 25837 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:14.452919 25853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:14.453142 25837 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:14.453310 25837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "fad188054df44188baa7435fa618e3be"
format_stamp: "Formatted at 2026-09-24 23:13:12 on dist-test-slave-m0z6"
server_key: "99a8d64ddd94239ebc91e310407c888d"
server_key_iv: "08fa38629efd23beb35d29024fe39d30"
server_key_version: "kuduclusterkey@0"
I20260924 23:13:14.453686 25837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:14.477905 25837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:14.479876 25837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:14.479996 25837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:14.484138 25837 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:43659
I20260924 23:13:14.484185 25905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:43659 every 8 connection(s)
I20260924 23:13:14.484573 25837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:13:14.485839 25906 sys_catalog.cc:263] Verifying existing consensus state
I20260924 23:13:14.486565 25906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be: Bootstrap starting.
I20260924 23:13:14.488943 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 25837
I20260924 23:13:14.489217 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.212.20:38651/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:42907
--local_ip_for_outbound_sockets=127.12.212.1
--tserver_master_addrs=127.12.212.62:43659
--webserver_port=34791
--webserver_interface=127.12.212.1
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.212.20:38651/kms with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 23:13:14.494499 25906 log.cc:824] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:14.496392 25906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 23:13:14.496677 25906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be: Bootstrap complete.
I20260924 23:13:14.498975 25906 raft_consensus.cc:348] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fad188054df44188baa7435fa618e3be" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 43659 } }
I20260924 23:13:14.499228 25906 raft_consensus.cc:729] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fad188054df44188baa7435fa618e3be, State: Initialized, Role: FOLLOWER
I20260924 23:13:14.499382 25906 consensus_queue.cc:260] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fad188054df44188baa7435fa618e3be" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 43659 } }
I20260924 23:13:14.499450 25906 raft_consensus.cc:388] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:13:14.499495 25906 raft_consensus.cc:482] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:13:14.499540 25906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [term 1 FOLLOWER]: Advancing to term 2
I20260924 23:13:14.500602 25906 raft_consensus.cc:504] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fad188054df44188baa7435fa618e3be" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 43659 } }
I20260924 23:13:14.500703 25906 leader_election.cc:302] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fad188054df44188baa7435fa618e3be; no voters:
I20260924 23:13:14.500886 25906 leader_election.cc:288] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 23:13:14.500939 25910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [term 2 FOLLOWER]: Leader election won for term 2
I20260924 23:13:14.501084 25910 raft_consensus.cc:686] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [term 2 LEADER]: Becoming Leader. State: Replica: fad188054df44188baa7435fa618e3be, State: Running, Role: LEADER
I20260924 23:13:14.501173 25906 sys_catalog.cc:564] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:13:14.501250 25910 consensus_queue.cc:237] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fad188054df44188baa7435fa618e3be" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 43659 } }
I20260924 23:13:14.501605 25910 sys_catalog.cc:455] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [sys.catalog]: SysCatalogTable state changed. Reason: New leader fad188054df44188baa7435fa618e3be. Latest consensus state: current_term: 2 leader_uuid: "fad188054df44188baa7435fa618e3be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fad188054df44188baa7435fa618e3be" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 43659 } } }
I20260924 23:13:14.501780 25911 sys_catalog.cc:455] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fad188054df44188baa7435fa618e3be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fad188054df44188baa7435fa618e3be" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 43659 } } }
I20260924 23:13:14.501852 25910 sys_catalog.cc:458] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [sys.catalog]: This master's current role is: LEADER
I20260924 23:13:14.502091 25911 sys_catalog.cc:458] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be [sys.catalog]: This master's current role is: LEADER
I20260924 23:13:14.502324 25919 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:13:14.502714 25906 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 23:13:14.502910 25919 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:13:14.503085 25919 catalog_manager.cc:1259] Loaded cluster ID: f76bff98391541dfbe2a6273a3213a4e
I20260924 23:13:14.503151 25919 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:13:14.507027 25919 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:13:14.507308 25919 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be: Loaded TSK: 0
I20260924 23:13:14.507543 25919 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 23:13:14.604837 25908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:14.605226 25908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:14.605357 25908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:14.607478 25908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:14.607550 25908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:14.607595 25908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:13:14.610124 25908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.212.20:38651/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:42907
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=34791
--tserver_master_addrs=127.12.212.62:43659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:14.610590 25908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:14.610960 25908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:14.617280 25943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:14.617343 25946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:14.617565 25944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:14.617735 25908 server_base.cc:1034] running on GCE node
I20260924 23:13:14.618018 25908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:14.618316 25908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:14.619606 25908 hybrid_clock.cc:648] HybridClock initialized: now 1790291594619573 us; error 45 us; skew 500 ppm
Sep 24 23:13:14 dist-test-slave-m0z6 krb5kdc[24874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291594, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:14.623025 25908 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:13:14.623610 25908 webserver.cc:460] Webserver started at http://127.12.212.1:34791/ using document root <none> and password file <none>
I20260924 23:13:14.623932 25908 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:14.624199 25908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 23:13:14 dist-test-slave-m0z6 krb5kdc[24874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291594, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
I20260924 23:13:14.647224 25908 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.000s sys 0.001s
I20260924 23:13:14.648411 25954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:14.648586 25908 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 23:13:14.648680 25908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "074eec832c8748c98fcdb51fbc80522e"
format_stamp: "Formatted at 2026-09-24 23:13:13 on dist-test-slave-m0z6"
server_key: "0eef6355b33f065413bb8347e2f63caf"
server_key_iv: "3824a6875775063847915dec9041eba6"
server_key_version: "kuduclusterkey@0"
I20260924 23:13:14.649175 25908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:14.671917 25908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:14.674419 25908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:14.674638 25908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:14.674985 25908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:13:14.675426 25908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:13:14.675518 25908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:14.675609 25908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:13:14.675674 25908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:14.682113 25908 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:42907
I20260924 23:13:14.682682 25908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:13:14.684970 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 25908
I20260924 23:13:14.685256 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.212.20:38651/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:41629
--local_ip_for_outbound_sockets=127.12.212.2
--tserver_master_addrs=127.12.212.62:43659
--webserver_port=36013
--webserver_interface=127.12.212.2
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.212.20:38651/kms with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 23:13:14.696291 26067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:42907 every 8 connection(s)
Sep 24 23:13:14 dist-test-slave-m0z6 krb5kdc[24874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291594, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:14.705736 26071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:14.685856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:47781 (local address 127.12.212.62:43659)
0924 23:13:14.686076 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:14.686088 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:14.687292 (+ 1204us) server_negotiation.cc:407] Connection header received
0924 23:13:14.690323 (+ 3031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:14.690332 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:14.690405 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:14.690546 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:14.691471 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:14.692908 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:14.694536 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:14.695251 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:14.695348 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:14.701085 (+ 5737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:14.701092 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:14.701092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:14.701109 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:14.703284 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:14.703960 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:14.703966 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:14.703966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:14.704108 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:14.704452 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:14.704454 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:14.704454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:14.704821 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:14.704901 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:14.705338 (+ 437us) server_negotiation.cc:299] Negotiation successful
0924 23:13:14.705494 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":90,"threads_started":1}
I20260924 23:13:14.707052 26070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:14.685819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:43659 (local address 127.12.212.1:47781)
0924 23:13:14.687006 (+ 1187us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:14.687077 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:14.690013 (+ 2936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:14.690767 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:14.690773 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:14.691022 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:14.691319 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:14.691326 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:14.693076 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:14.693077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:14.694413 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:14.694418 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:14.695516 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:14.695517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:14.695673 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:14.696533 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:14.696545 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:14.700903 (+ 4358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:14.703417 (+ 2514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:14.703430 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:14.703434 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:14.703832 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:14.704193 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:14.704194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:14.704194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:14.704366 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:14.704940 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:14.704948 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:14.705123 (+ 175us) client_negotiation.cc:769] Sending connection context
0924 23:13:14.706555 (+ 1432us) client_negotiation.cc:240] Negotiation successful
0924 23:13:14.706806 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1122,"thread_start_us":89,"threads_started":1}
I20260924 23:13:14.708292 26068 heartbeater.cc:347] Connected to a master server at 127.12.212.62:43659
I20260924 23:13:14.708397 26068 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:14.708601 26068 heartbeater.cc:511] Master 127.12.212.62:43659 requested a full tablet report, sending...
W20260924 23:13:14.709582 26068 heartbeater.cc:644] Failed to heartbeat to 127.12.212.62:43659 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 23:13:14.709659 26068 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:14.709728 26068 heartbeater.cc:511] Master 127.12.212.62:43659 requested a full tablet report, sending...
I20260924 23:13:14.710222 26068 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:14.710302 26068 heartbeater.cc:511] Master 127.12.212.62:43659 requested a full tablet report, sending...
W20260924 23:13:14.710668 26068 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 23:13:14.793987 26072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:14.794198 26072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:14.794242 26072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:14.795727 26072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:14.795856 26072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:14.795948 26072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:13:14.797622 26072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.212.20:38651/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:41629
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=36013
--tserver_master_addrs=127.12.212.62:43659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:14.798110 26072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:14.798458 26072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:14.803660 26078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:14.803987 26077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:14.804664 26080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:14.804909 26072 server_base.cc:1034] running on GCE node
I20260924 23:13:14.805114 26072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:14.805372 26072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:14.806523 26072 hybrid_clock.cc:648] HybridClock initialized: now 1790291594806507 us; error 35 us; skew 500 ppm
Sep 24 23:13:14 dist-test-slave-m0z6 krb5kdc[24874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291594, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:14.809679 26072 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:13:14.810266 26072 webserver.cc:460] Webserver started at http://127.12.212.2:36013/ using document root <none> and password file <none>
I20260924 23:13:14.810510 26072 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:14.810576 26072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 23:13:14 dist-test-slave-m0z6 krb5kdc[24874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291594, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
I20260924 23:13:14.830295 26072 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:13:14.831230 26088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:14.831415 26072 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 23:13:14.831509 26072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "aca7815dbeb54667ae2af17dfaeb500b"
format_stamp: "Formatted at 2026-09-24 23:13:13 on dist-test-slave-m0z6"
server_key: "b3c9bc7a0cb5607187b19fee59f4f9ea"
server_key_iv: "52143bec96fb6018d20c620575b9b81e"
server_key_version: "kuduclusterkey@0"
I20260924 23:13:14.831905 26072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:14.842607 26072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:14.845212 26072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:14.845391 26072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:14.845634 26072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:13:14.845999 26072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:13:14.846035 26072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:14.846056 26072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:13:14.846067 26072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:14.852736 26072 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:41629
I20260924 23:13:14.852821 26201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:41629 every 8 connection(s)
I20260924 23:13:14.853211 26072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:13:14.854228 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 26072
I20260924 23:13:14.854470 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.212.20:38651/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:36249
--local_ip_for_outbound_sockets=127.12.212.3
--tserver_master_addrs=127.12.212.62:43659
--webserver_port=39775
--webserver_interface=127.12.212.3
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.212.20:38651/kms with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 23:13:14 dist-test-slave-m0z6 krb5kdc[24874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291594, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:14.864894 26071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:14.853924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:59755 (local address 127.12.212.62:43659)
0924 23:13:14.854020 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:14.854022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:14.854218 (+ 196us) server_negotiation.cc:407] Connection header received
0924 23:13:14.854947 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:14.854948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:14.854964 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:14.855024 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:14.855793 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:14.856593 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:14.858110 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:14.858429 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:14.858509 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:14.861214 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:14.861217 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:14.861217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:14.861226 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:14.863039 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:14.863653 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:14.863655 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:14.863656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:14.863728 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:14.864074 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:14.864076 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:14.864076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:14.864231 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:14.864316 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:14.864692 (+ 376us) server_negotiation.cc:299] Negotiation successful
0924 23:13:14.864791 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 23:13:14.865098 26204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:14.853899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:43659 (local address 127.12.212.2:59755)
0924 23:13:14.854054 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:14.854107 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:14.854830 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:14.855122 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:14.855127 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:14.855376 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:14.855669 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:14.855676 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:14.856734 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:14.856735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:14.858014 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:14.858017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:14.858554 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:14.858555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:14.858734 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:14.859362 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:14.859372 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:14.861083 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:14.863149 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:14.863162 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:14.863165 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:14.863548 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:14.863802 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:14.863803 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:14.863803 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:14.863996 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:14.864315 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:14.864323 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:14.864521 (+ 198us) client_negotiation.cc:769] Sending connection context
0924 23:13:14.864660 (+ 139us) client_negotiation.cc:240] Negotiation successful
0924 23:13:14.864843 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
I20260924 23:13:14.865684 26202 heartbeater.cc:347] Connected to a master server at 127.12.212.62:43659
I20260924 23:13:14.865789 26202 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:14.865984 26202 heartbeater.cc:511] Master 127.12.212.62:43659 requested a full tablet report, sending...
W20260924 23:13:14.866639 26202 heartbeater.cc:644] Failed to heartbeat to 127.12.212.62:43659 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 23:13:14.866737 26202 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:14.866799 26202 heartbeater.cc:511] Master 127.12.212.62:43659 requested a full tablet report, sending...
I20260924 23:13:14.867202 26202 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:14.867272 26202 heartbeater.cc:511] Master 127.12.212.62:43659 requested a full tablet report, sending...
W20260924 23:13:14.867635 26202 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 23:13:14.958207 26205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:14.958446 26205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:14.958484 26205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:14.959954 26205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:14.960009 26205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:14.960052 26205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:13:14.961879 26205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.212.20:38651/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:36249
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=39775
--tserver_master_addrs=127.12.212.62:43659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:14.962162 26205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:14.962421 26205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:14.968130 26213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:14.970244 26205 server_base.cc:1034] running on GCE node
W20260924 23:13:14.971000 26211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:14.971379 26210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:14.971692 26205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:14.972008 26205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:14.973212 26205 hybrid_clock.cc:648] HybridClock initialized: now 1790291594973190 us; error 36 us; skew 500 ppm
Sep 24 23:13:14 dist-test-slave-m0z6 krb5kdc[24874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291594, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:14.976001 26205 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:13:14.976516 26205 webserver.cc:460] Webserver started at http://127.12.212.3:39775/ using document root <none> and password file <none>
I20260924 23:13:14.976781 26205 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:14.976861 26205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 23:13:14 dist-test-slave-m0z6 krb5kdc[24874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291594, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
I20260924 23:13:14.998392 26205 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260924 23:13:15.007270 26222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:15.007609 26205 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.000s sys 0.000s
I20260924 23:13:15.007767 26205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "36ce009abb1b4461b3afacacd6965959"
format_stamp: "Formatted at 2026-09-24 23:13:14 on dist-test-slave-m0z6"
server_key: "57144c06fbb5cfcb162993dd53ba45bb"
server_key_iv: "d0a5eb6f099b8bfa0fbe5e12a2537755"
server_key_version: "kuduclusterkey@0"
I20260924 23:13:15.008396 26205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:15.039157 26205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:15.041332 26205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:15.041517 26205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:15.041838 26205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:13:15.042455 26205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:13:15.042503 26205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:15.042528 26205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:13:15.042542 26205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:15.049602 26205 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:36249
I20260924 23:13:15.050115 26205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:13:15.050539 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 26205
I20260924 23:13:15.050626 26335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:36249 every 8 connection(s)
I20260924 23:13:15.060413 26071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:15.051044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47852 (local address 127.12.212.62:43659)
0924 23:13:15.051099 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:15.051101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:15.051263 (+ 162us) server_negotiation.cc:407] Connection header received
0924 23:13:15.051924 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:15.051926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:15.051940 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:15.051989 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:15.052740 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:15.053505 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:15.054738 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:15.055055 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:15.055106 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:15.056877 (+ 1771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:15.056879 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:15.056879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:15.056889 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:15.058532 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:15.058947 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:15.058949 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:15.058949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:15.059013 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:15.059753 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:15.059754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:15.059754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:15.059927 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:15.060129 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:15.060180 (+ 51us) server_negotiation.cc:299] Negotiation successful
0924 23:13:15.060309 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Sep 24 23:13:15 dist-test-slave-m0z6 krb5kdc[24874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291594, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:15.068638 26339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:15.051594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:57635 (local address 127.12.212.62:43659)
0924 23:13:15.051726 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:15.051728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:15.057456 (+ 5728us) server_negotiation.cc:407] Connection header received
0924 23:13:15.058513 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:15.058514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:15.058531 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:15.058591 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:15.059565 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:15.060188 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:15.061841 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:15.062182 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:15.062265 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:15.064763 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:15.064765 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:15.064766 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:15.064774 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:15.066671 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:15.067323 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:15.067325 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:15.067325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:15.067397 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:15.067746 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:15.067747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:15.067748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:15.067923 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:15.068001 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:15.068424 (+ 423us) server_negotiation.cc:299] Negotiation successful
0924 23:13:15.068527 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260924 23:13:15.068815 26340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:15.051605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:43659 (local address 127.12.212.3:57635)
0924 23:13:15.056954 (+ 5349us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:15.057037 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:15.058309 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:15.059075 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:15.059081 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:15.059260 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:15.059474 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:15.059480 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:15.060306 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:15.060307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:15.061533 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:15.061536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:15.062380 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:15.062381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:15.062506 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:15.063105 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:15.063116 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:15.064602 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:15.066832 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:15.066842 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:15.066845 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:15.067203 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:15.067482 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:15.067483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:15.067483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:15.067656 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:15.067996 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:15.068004 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:15.068221 (+ 217us) client_negotiation.cc:769] Sending connection context
0924 23:13:15.068357 (+ 136us) client_negotiation.cc:240] Negotiation successful
0924 23:13:15.068550 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5271,"thread_start_us":80,"threads_started":1}
I20260924 23:13:15.069331 26336 heartbeater.cc:347] Connected to a master server at 127.12.212.62:43659
I20260924 23:13:15.069444 26336 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:15.069638 26336 heartbeater.cc:511] Master 127.12.212.62:43659 requested a full tablet report, sending...
W20260924 23:13:15.070410 26336 heartbeater.cc:644] Failed to heartbeat to 127.12.212.62:43659 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 23:13:15.070511 26336 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:15.070592 26336 heartbeater.cc:511] Master 127.12.212.62:43659 requested a full tablet report, sending...
I20260924 23:13:15.071022 26336 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:15.071117 26336 heartbeater.cc:511] Master 127.12.212.62:43659 requested a full tablet report, sending...
W20260924 23:13:15.071560 26336 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 24 23:13:15 dist-test-slave-m0z6 krb5kdc[24874](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790291595, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:15.710898 26068 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:15.711088 26068 heartbeater.cc:511] Master 127.12.212.62:43659 requested a full tablet report, sending...
I20260924 23:13:15.711520 25870 ts_manager.cc:194] Registered new tserver with Master: 074eec832c8748c98fcdb51fbc80522e (127.12.212.1:42907)
I20260924 23:13:15.712574 25870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:47781
I20260924 23:13:15.867800 26202 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:15.867987 26202 heartbeater.cc:511] Master 127.12.212.62:43659 requested a full tablet report, sending...
I20260924 23:13:15.870313 25870 ts_manager.cc:194] Registered new tserver with Master: aca7815dbeb54667ae2af17dfaeb500b (127.12.212.2:41629)
I20260924 23:13:15.870980 25870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:59755
Sep 24 23:13:16 dist-test-slave-m0z6 krb5kdc[24874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291595, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for HTTP/127.12.212.20@KRBTEST.COM
I20260924 23:13:16.071749 26336 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:16.071918 26336 heartbeater.cc:511] Master 127.12.212.62:43659 requested a full tablet report, sending...
I20260924 23:13:16.072358 25870 ts_manager.cc:194] Registered new tserver with Master: 36ce009abb1b4461b3afacacd6965959 (127.12.212.3:36249)
I20260924 23:13:16.072886 25870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:57635
I20260924 23:13:16.074452 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 23:13:16.090950 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:16.079268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47856 (local address 127.12.212.62:43659)
0924 23:13:16.080619 (+ 1351us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:16.080622 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:16.082867 (+ 2245us) server_negotiation.cc:407] Connection header received
0924 23:13:16.082940 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:16.082941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:16.082957 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:16.083056 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:16.083806 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.084610 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.085781 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.086110 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.086177 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:16.087795 (+ 1618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:16.087797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:16.087797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:16.087805 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:16.089600 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:16.090042 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:16.090043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:16.090044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:16.090112 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:16.090341 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:16.090342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:16.090342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:16.090489 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:16.090590 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:16.090746 (+ 156us) server_negotiation.cc:299] Negotiation successful
0924 23:13:16.090838 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1322,"thread_start_us":99,"threads_started":1}
I20260924 23:13:16.093499 25870 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:13:16.096086 25870 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:13:16.104969 26365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:16.099458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:42907 (local address 127.0.0.1:40096)
0924 23:13:16.100114 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:16.100158 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:16.100225 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:16.100662 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:16.100665 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:16.100667 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:13:16.100738 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:16.100740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.102041 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.102042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:16.103619 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:16.103622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.104656 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.104657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:16.104736 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:16.104750 (+ 14us) client_negotiation.cc:769] Sending connection context
0924 23:13:16.104817 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 23:13:16.104872 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":104,"threads_started":1}
I20260924 23:13:16.105279 26367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:16.099968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40096 (local address 127.12.212.1:42907)
0924 23:13:16.100351 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:16.100356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:16.100366 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:13:16.100399 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:16.100401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:16.100417 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:16.100582 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:13:16.100806 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.101958 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.103722 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.104481 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.104561 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:16.104621 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:16.105148 (+ 527us) server_negotiation.cc:299] Negotiation successful
0924 23:13:16.105198 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":46,"threads_started":1}
I20260924 23:13:16.106292 25999 tablet_service.cc:1467] Processing CreateTablet for tablet a7856705ba7f433abcbfa0fd02db87d7 (DEFAULT_TABLE table=test-table [id=f0cb933968784bd38d4036acd11f8980]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:13:16.106729 25999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7856705ba7f433abcbfa0fd02db87d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:16.113297 26365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:16.099709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:41629 (local address 127.0.0.1:42208)
0924 23:13:16.105447 (+ 5738us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:16.105459 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:16.105500 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:16.109258 (+ 3758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:16.109259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:16.109261 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:13:16.109306 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:16.109308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.110512 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.110513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:16.111585 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:16.111588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.113034 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.113035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:16.113105 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:16.113108 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:13:16.113168 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 23:13:16.113212 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5726,"thread_start_us":59,"threads_started":1}
I20260924 23:13:16.113334 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:16.099981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42208 (local address 127.12.212.2:41629)
0924 23:13:16.109022 (+ 9041us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:16.109026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:16.109036 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:13:16.109068 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:16.109070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:16.109085 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:16.109161 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:13:16.109382 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.110413 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.111685 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.112444 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.112516 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:16.112581 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:16.113210 (+ 629us) server_negotiation.cc:299] Negotiation successful
0924 23:13:16.113254 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9002,"thread_start_us":60,"threads_started":1}
I20260924 23:13:16.114419 26118 tablet_service.cc:1467] Processing CreateTablet for tablet a7856705ba7f433abcbfa0fd02db87d7 (DEFAULT_TABLE table=test-table [id=f0cb933968784bd38d4036acd11f8980]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:13:16.114807 26118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7856705ba7f433abcbfa0fd02db87d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:16.118444 26370 tablet_bootstrap.cc:492] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b: Bootstrap starting.
I20260924 23:13:16.119241 26370 tablet_bootstrap.cc:654] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b: Neither blocks nor log segments found. Creating new log.
I20260924 23:13:16.119652 26370 log.cc:824] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:16.120538 26370 tablet_bootstrap.cc:492] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b: No bootstrap required, opened a new log
I20260924 23:13:16.120602 26370 ts_tablet_manager.cc:1397] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:13:16.122457 26363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:16.099458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:36249 (local address 127.0.0.1:39592)
0924 23:13:16.102781 (+ 3323us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:16.102792 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:16.102862 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:16.118778 (+ 15916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:16.118780 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:16.118783 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:13:16.118850 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:16.118852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.120088 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.120090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:16.121166 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:16.121170 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.122197 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.122198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:16.122265 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:16.122270 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 23:13:16.122312 (+ 42us) client_negotiation.cc:240] Negotiation successful
0924 23:13:16.122365 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3305,"mutex_wait_us":3,"thread_start_us":104,"threads_started":1}
I20260924 23:13:16.122788 26366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:16.099739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39592 (local address 127.12.212.3:36249)
0924 23:13:16.108696 (+ 8957us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:16.108703 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:16.108715 (+ 12us) server_negotiation.cc:407] Connection header received
0924 23:13:16.108754 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:16.108756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:16.108773 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:16.108881 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:13:16.118949 (+ 10068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.119981 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.121273 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.122028 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.122098 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:16.122158 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:16.122638 (+ 480us) server_negotiation.cc:299] Negotiation successful
0924 23:13:16.122712 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8915,"thread_start_us":68,"threads_started":1}
I20260924 23:13:16.123124 26370 raft_consensus.cc:348] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce009abb1b4461b3afacacd6965959" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36249 } } peers { permanent_uuid: "074eec832c8748c98fcdb51fbc80522e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42907 } } peers { permanent_uuid: "aca7815dbeb54667ae2af17dfaeb500b" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 41629 } }
I20260924 23:13:16.123291 26370 raft_consensus.cc:374] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:13:16.123258 26369 tablet_bootstrap.cc:492] T a7856705ba7f433abcbfa0fd02db87d7 P 074eec832c8748c98fcdb51fbc80522e: Bootstrap starting.
I20260924 23:13:16.123320 26370 raft_consensus.cc:729] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aca7815dbeb54667ae2af17dfaeb500b, State: Initialized, Role: FOLLOWER
I20260924 23:13:16.123520 26251 tablet_service.cc:1467] Processing CreateTablet for tablet a7856705ba7f433abcbfa0fd02db87d7 (DEFAULT_TABLE table=test-table [id=f0cb933968784bd38d4036acd11f8980]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:13:16.123910 26251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7856705ba7f433abcbfa0fd02db87d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:16.123997 26369 tablet_bootstrap.cc:654] T a7856705ba7f433abcbfa0fd02db87d7 P 074eec832c8748c98fcdb51fbc80522e: Neither blocks nor log segments found. Creating new log.
I20260924 23:13:16.124308 26369 log.cc:824] T a7856705ba7f433abcbfa0fd02db87d7 P 074eec832c8748c98fcdb51fbc80522e: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:16.124372 26370 consensus_queue.cc:260] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce009abb1b4461b3afacacd6965959" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36249 } } peers { permanent_uuid: "074eec832c8748c98fcdb51fbc80522e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42907 } } peers { permanent_uuid: "aca7815dbeb54667ae2af17dfaeb500b" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 41629 } }
I20260924 23:13:16.124691 26370 ts_tablet_manager.cc:1428] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260924 23:13:16.124723 26202 heartbeater.cc:503] Master 127.12.212.62:43659 was elected leader, sending a full tablet report...
I20260924 23:13:16.126259 26369 tablet_bootstrap.cc:492] T a7856705ba7f433abcbfa0fd02db87d7 P 074eec832c8748c98fcdb51fbc80522e: No bootstrap required, opened a new log
I20260924 23:13:16.126397 26369 ts_tablet_manager.cc:1397] T a7856705ba7f433abcbfa0fd02db87d7 P 074eec832c8748c98fcdb51fbc80522e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 23:13:16.127823 26374 tablet_bootstrap.cc:492] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959: Bootstrap starting.
I20260924 23:13:16.128602 26374 tablet_bootstrap.cc:654] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959: Neither blocks nor log segments found. Creating new log.
I20260924 23:13:16.128988 26374 log.cc:824] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:16.128952 26369 raft_consensus.cc:348] T a7856705ba7f433abcbfa0fd02db87d7 P 074eec832c8748c98fcdb51fbc80522e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce009abb1b4461b3afacacd6965959" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36249 } } peers { permanent_uuid: "074eec832c8748c98fcdb51fbc80522e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42907 } } peers { permanent_uuid: "aca7815dbeb54667ae2af17dfaeb500b" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 41629 } }
I20260924 23:13:16.129221 26369 raft_consensus.cc:374] T a7856705ba7f433abcbfa0fd02db87d7 P 074eec832c8748c98fcdb51fbc80522e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:13:16.129303 26369 raft_consensus.cc:729] T a7856705ba7f433abcbfa0fd02db87d7 P 074eec832c8748c98fcdb51fbc80522e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 074eec832c8748c98fcdb51fbc80522e, State: Initialized, Role: FOLLOWER
I20260924 23:13:16.129511 26369 consensus_queue.cc:260] T a7856705ba7f433abcbfa0fd02db87d7 P 074eec832c8748c98fcdb51fbc80522e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce009abb1b4461b3afacacd6965959" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36249 } } peers { permanent_uuid: "074eec832c8748c98fcdb51fbc80522e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42907 } } peers { permanent_uuid: "aca7815dbeb54667ae2af17dfaeb500b" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 41629 } }
I20260924 23:13:16.129869 26369 ts_tablet_manager.cc:1428] T a7856705ba7f433abcbfa0fd02db87d7 P 074eec832c8748c98fcdb51fbc80522e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 23:13:16.130028 26068 heartbeater.cc:503] Master 127.12.212.62:43659 was elected leader, sending a full tablet report...
I20260924 23:13:16.131510 26374 tablet_bootstrap.cc:492] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959: No bootstrap required, opened a new log
I20260924 23:13:16.131593 26374 ts_tablet_manager.cc:1397] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260924 23:13:16.133921 26374 raft_consensus.cc:348] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce009abb1b4461b3afacacd6965959" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36249 } } peers { permanent_uuid: "074eec832c8748c98fcdb51fbc80522e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42907 } } peers { permanent_uuid: "aca7815dbeb54667ae2af17dfaeb500b" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 41629 } }
I20260924 23:13:16.134128 26374 raft_consensus.cc:374] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:13:16.134163 26374 raft_consensus.cc:729] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36ce009abb1b4461b3afacacd6965959, State: Initialized, Role: FOLLOWER
I20260924 23:13:16.134349 26374 consensus_queue.cc:260] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce009abb1b4461b3afacacd6965959" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36249 } } peers { permanent_uuid: "074eec832c8748c98fcdb51fbc80522e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42907 } } peers { permanent_uuid: "aca7815dbeb54667ae2af17dfaeb500b" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 41629 } }
I20260924 23:13:16.134757 26374 ts_tablet_manager.cc:1428] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:13:16.134866 26336 heartbeater.cc:503] Master 127.12.212.62:43659 was elected leader, sending a full tablet report...
W20260924 23:13:16.186666 26069 tablet.cc:2369] T a7856705ba7f433abcbfa0fd02db87d7 P 074eec832c8748c98fcdb51fbc80522e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:13:16.214795 26377 raft_consensus.cc:482] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:13:16.214974 26377 raft_consensus.cc:504] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce009abb1b4461b3afacacd6965959" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36249 } } peers { permanent_uuid: "074eec832c8748c98fcdb51fbc80522e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42907 } } peers { permanent_uuid: "aca7815dbeb54667ae2af17dfaeb500b" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 41629 } }
I20260924 23:13:16.215405 26377 leader_election.cc:288] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 074eec832c8748c98fcdb51fbc80522e (127.12.212.1:42907), aca7815dbeb54667ae2af17dfaeb500b (127.12.212.2:41629)
I20260924 23:13:16.219888 26367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:16.215916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:33325 (local address 127.12.212.1:42907)
0924 23:13:16.216012 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:16.216014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:16.216021 (+ 7us) server_negotiation.cc:407] Connection header received
0924 23:13:16.216046 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:16.216047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:16.216056 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:16.216094 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:13:16.216276 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.216978 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.218917 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.219465 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.219520 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:16.219530 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:16.219763 (+ 233us) server_negotiation.cc:299] Negotiation successful
0924 23:13:16.219812 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260924 23:13:16.219956 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:16.215982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:42447 (local address 127.12.212.2:41629)
0924 23:13:16.216024 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:16.216026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:16.216151 (+ 125us) server_negotiation.cc:407] Connection header received
0924 23:13:16.216245 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:16.216247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:16.216255 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:16.216283 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:13:16.216500 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.217293 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.219027 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.219559 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.219613 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:16.219623 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:16.219839 (+ 216us) server_negotiation.cc:299] Negotiation successful
0924 23:13:16.219885 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 23:13:16.220202 26379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:16.215680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:41629 (local address 127.12.212.3:42447)
0924 23:13:16.216089 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:16.216098 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:16.216175 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:16.216348 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:16.216348 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:16.216350 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:13:16.216420 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:16.216422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.217635 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.217636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:16.218916 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:16.218919 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.219680 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.219680 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:16.219752 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:16.219768 (+ 16us) client_negotiation.cc:769] Sending connection context
0924 23:13:16.220070 (+ 302us) client_negotiation.cc:240] Negotiation successful
0924 23:13:16.220116 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":104,"threads_started":1}
I20260924 23:13:16.220490 26378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:16.215583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:42907 (local address 127.12.212.3:33325)
0924 23:13:16.215852 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:16.215863 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:16.215922 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:16.216137 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:16.216138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:16.216140 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:13:16.216213 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:16.216215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.217073 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.217074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:16.218818 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:16.218822 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.219609 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.219610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:16.219680 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:16.219685 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 23:13:16.220353 (+ 668us) client_negotiation.cc:240] Negotiation successful
0924 23:13:16.220409 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":107,"threads_started":1}
I20260924 23:13:16.220629 26156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7856705ba7f433abcbfa0fd02db87d7" candidate_uuid: "36ce009abb1b4461b3afacacd6965959" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aca7815dbeb54667ae2af17dfaeb500b" is_pre_election: true
I20260924 23:13:16.220825 26156 raft_consensus.cc:2454] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36ce009abb1b4461b3afacacd6965959 in term 0.
I20260924 23:13:16.220960 26010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7856705ba7f433abcbfa0fd02db87d7" candidate_uuid: "36ce009abb1b4461b3afacacd6965959" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "074eec832c8748c98fcdb51fbc80522e" is_pre_election: true
I20260924 23:13:16.221069 26225 leader_election.cc:302] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36ce009abb1b4461b3afacacd6965959, aca7815dbeb54667ae2af17dfaeb500b; no voters:
I20260924 23:13:16.221105 26010 raft_consensus.cc:2454] T a7856705ba7f433abcbfa0fd02db87d7 P 074eec832c8748c98fcdb51fbc80522e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36ce009abb1b4461b3afacacd6965959 in term 0.
I20260924 23:13:16.221350 26377 raft_consensus.cc:2781] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:13:16.221412 26377 raft_consensus.cc:482] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:13:16.221433 26377 raft_consensus.cc:3037] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:13:16.222332 26377 raft_consensus.cc:504] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce009abb1b4461b3afacacd6965959" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36249 } } peers { permanent_uuid: "074eec832c8748c98fcdb51fbc80522e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42907 } } peers { permanent_uuid: "aca7815dbeb54667ae2af17dfaeb500b" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 41629 } }
I20260924 23:13:16.222743 26010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7856705ba7f433abcbfa0fd02db87d7" candidate_uuid: "36ce009abb1b4461b3afacacd6965959" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "074eec832c8748c98fcdb51fbc80522e"
I20260924 23:13:16.222842 26010 raft_consensus.cc:3037] T a7856705ba7f433abcbfa0fd02db87d7 P 074eec832c8748c98fcdb51fbc80522e [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:13:16.223567 26377 leader_election.cc:288] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [CANDIDATE]: Term 1 election: Requested vote from peers 074eec832c8748c98fcdb51fbc80522e (127.12.212.1:42907), aca7815dbeb54667ae2af17dfaeb500b (127.12.212.2:41629)
I20260924 23:13:16.223618 26010 raft_consensus.cc:2454] T a7856705ba7f433abcbfa0fd02db87d7 P 074eec832c8748c98fcdb51fbc80522e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36ce009abb1b4461b3afacacd6965959 in term 1.
I20260924 23:13:16.223856 26226 leader_election.cc:302] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 074eec832c8748c98fcdb51fbc80522e, 36ce009abb1b4461b3afacacd6965959; no voters:
I20260924 23:13:16.223920 26156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7856705ba7f433abcbfa0fd02db87d7" candidate_uuid: "36ce009abb1b4461b3afacacd6965959" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aca7815dbeb54667ae2af17dfaeb500b"
I20260924 23:13:16.224013 26377 raft_consensus.cc:2781] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:13:16.224009 26156 raft_consensus.cc:3037] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:13:16.224165 26377 raft_consensus.cc:686] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [term 1 LEADER]: Becoming Leader. State: Replica: 36ce009abb1b4461b3afacacd6965959, State: Running, Role: LEADER
I20260924 23:13:16.224263 26377 consensus_queue.cc:237] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce009abb1b4461b3afacacd6965959" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36249 } } peers { permanent_uuid: "074eec832c8748c98fcdb51fbc80522e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42907 } } peers { permanent_uuid: "aca7815dbeb54667ae2af17dfaeb500b" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 41629 } }
I20260924 23:13:16.224911 26156 raft_consensus.cc:2454] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36ce009abb1b4461b3afacacd6965959 in term 1.
I20260924 23:13:16.225425 25870 catalog_manager.cc:5495] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36ce009abb1b4461b3afacacd6965959 (127.12.212.3). New cstate: current_term: 1 leader_uuid: "36ce009abb1b4461b3afacacd6965959" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ce009abb1b4461b3afacacd6965959" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "074eec832c8748c98fcdb51fbc80522e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aca7815dbeb54667ae2af17dfaeb500b" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 41629 } health_report { overall_health: UNKNOWN } } }
I20260924 23:13:16.252404 26366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:16.249381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39600 (local address 127.12.212.3:36249)
0924 23:13:16.249454 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:16.249456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:16.249464 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:13:16.249503 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:16.249504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:16.249525 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:16.249592 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:13:16.249773 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.250477 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.251620 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:16.251827 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:16.251887 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:16.252060 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:13:16.252186 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:13:16.252273 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:16.252300 (+ 27us) server_negotiation.cc:299] Negotiation successful
0924 23:13:16.252324 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260924 23:13:16.255002 26156 raft_consensus.cc:1261] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b [term 1 FOLLOWER]: Refusing update from remote peer 36ce009abb1b4461b3afacacd6965959: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:13:16.255300 26010 raft_consensus.cc:1261] T a7856705ba7f433abcbfa0fd02db87d7 P 074eec832c8748c98fcdb51fbc80522e [term 1 FOLLOWER]: Refusing update from remote peer 36ce009abb1b4461b3afacacd6965959: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:13:16.255503 26377 consensus_queue.cc:1035] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aca7815dbeb54667ae2af17dfaeb500b" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 41629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:13:16.255662 26377 consensus_queue.cc:1035] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [LEADER]: Connected to new peer: Peer: permanent_uuid: "074eec832c8748c98fcdb51fbc80522e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:13:16.260061 26383 mvcc.cc:205] Tried to move back new op lower bound from 7333034378256678912 to 7333034378135199744. Current Snapshot: MvccSnapshot[applied={T|T < 7333034378256678912}]
I20260924 23:13:16.261767 26384 mvcc.cc:205] Tried to move back new op lower bound from 7333034378256678912 to 7333034378135199744. Current Snapshot: MvccSnapshot[applied={T|T < 7333034378256678912}]
I20260924 23:13:16.262492 26385 mvcc.cc:205] Tried to move back new op lower bound from 7333034378256678912 to 7333034378135199744. Current Snapshot: MvccSnapshot[applied={T|T < 7333034378256678912}]
I20260924 23:13:16.263561 25870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:13:16.263669 25870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:13:16.264441 25870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fad188054df44188baa7435fa618e3be: Sending DeleteTablet for 3 replicas of tablet a7856705ba7f433abcbfa0fd02db87d7
I20260924 23:13:16.264894 26118 tablet_service.cc:1514] Processing DeleteTablet for tablet a7856705ba7f433abcbfa0fd02db87d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:13:16 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:42208
I20260924 23:13:16.265476 26251 tablet_service.cc:1514] Processing DeleteTablet for tablet a7856705ba7f433abcbfa0fd02db87d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:13:16 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:39592
I20260924 23:13:16.266345 25999 tablet_service.cc:1514] Processing DeleteTablet for tablet a7856705ba7f433abcbfa0fd02db87d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:13:16 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:40096
I20260924 23:13:16.266961 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 25908
I20260924 23:13:16.270831 26393 tablet_replica.cc:330] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959: stopping tablet replica
I20260924 23:13:16.270977 26393 raft_consensus.cc:2229] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:13:16.271093 26393 raft_consensus.cc:2258] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:13:16.271700 26393 ts_tablet_manager.cc:1905] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:13:16.272684 26392 tablet_replica.cc:330] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b: stopping tablet replica
I20260924 23:13:16.272879 26392 raft_consensus.cc:2229] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:13:16.273090 26392 raft_consensus.cc:2258] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:13:16.273670 26392 ts_tablet_manager.cc:1905] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:13:16.275389 26392 ts_tablet_manager.cc:1918] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:13:16.275524 26392 log.cc:1196] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/a7856705ba7f433abcbfa0fd02db87d7
I20260924 23:13:16.275838 26392 ts_tablet_manager.cc:1939] T a7856705ba7f433abcbfa0fd02db87d7 P aca7815dbeb54667ae2af17dfaeb500b: Deleting consensus metadata
I20260924 23:13:16.276374 25856 catalog_manager.cc:4841] TS aca7815dbeb54667ae2af17dfaeb500b (127.12.212.2:41629): tablet a7856705ba7f433abcbfa0fd02db87d7 (table test-table [id=f0cb933968784bd38d4036acd11f8980]) successfully deleted
W20260924 23:13:16.279796 25857 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.212.1:42907 (error 108)
I20260924 23:13:16.280650 26363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:16.280517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:42907 (local address 127.0.0.1:40110)
0924 23:13:16.280575 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:16.280610 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.1:42907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260924 23:13:16.280900 25857 catalog_manager.cc:4584] TS 074eec832c8748c98fcdb51fbc80522e (127.12.212.1:42907): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a7856705ba7f433abcbfa0fd02db87d7: Network error: Client connection negotiation failed: client connection to 127.12.212.1:42907: connect: Connection refused (error 111)
I20260924 23:13:16.282939 26393 ts_tablet_manager.cc:1918] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:13:16.283104 26393 log.cc:1196] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/a7856705ba7f433abcbfa0fd02db87d7
I20260924 23:13:16.283468 26393 ts_tablet_manager.cc:1939] T a7856705ba7f433abcbfa0fd02db87d7 P 36ce009abb1b4461b3afacacd6965959: Deleting consensus metadata
I20260924 23:13:16.285094 25855 catalog_manager.cc:4841] TS 36ce009abb1b4461b3afacacd6965959 (127.12.212.3:36249): tablet a7856705ba7f433abcbfa0fd02db87d7 (table test-table [id=f0cb933968784bd38d4036acd11f8980]) successfully deleted
I20260924 23:13:16.285418 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 26072
I20260924 23:13:16.293596 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 26205
I20260924 23:13:16.299712 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 25837
I20260924 23:13:16.306106 13136 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260924 23:13:16.641506 13136 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260924 23:13:16.641610 13136 mini_ranger.cc:66] Stopping Ranger...
I20260924 23:13:16.986920 13136 mini_ranger.cc:68] Stopped Ranger
2026-09-24 23:13:16.987 UTC [24904] LOG: received smart shutdown request
2026-09-24 23:13:16.991 UTC [24904] LOG: background worker "logical replication launcher" (PID 24913) exited with exit code 1
2026-09-24 23:13:16.991 UTC [24908] LOG: shutting down
2026-09-24 23:13:17.200 UTC [24904] LOG: database system is shut down
2026-09-24T23:13:17Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50147 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:13:17 dist-test-slave-m0z6 krb5kdc[26405](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:13:17 dist-test-slave-m0z6 krb5kdc[26405](info): set up 2 sockets
Sep 24 23:13:17 dist-test-slave-m0z6 krb5kdc[26405](info): commencing operation
krb5kdc: starting...
W20260924 23:13:18.254390 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.945s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:13:18 dist-test-slave-m0z6 krb5kdc[26405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:13:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:13:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:13:18.379757 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:37955
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:33585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:37955,127.12.212.61:40475,127.12.212.60:42285
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:13:18.463320 26418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:18.463558 26418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:18.463582 26418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:18.464865 26418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:13:18.464912 26418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:18.464926 26418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:18.464937 26418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:13:18.464948 26418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:13:18.466538 26418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:33585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:37955,127.12.212.61:40475,127.12.212.60:42285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:37955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:18.466821 26418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:18.467063 26418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:18.471957 26423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:18.471957 26424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:18.472124 26418 server_base.cc:1034] running on GCE node
W20260924 23:13:18.471967 26426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:18.472450 26418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:18.472754 26418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:18.473927 26418 hybrid_clock.cc:648] HybridClock initialized: now 1790291598473913 us; error 30 us; skew 500 ppm
Sep 24 23:13:18 dist-test-slave-m0z6 krb5kdc[26405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291598, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:18.481439 26418 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:13:18.482038 26418 webserver.cc:460] Webserver started at http://127.12.212.62:38557/ using document root <none> and password file <none>
I20260924 23:13:18.482275 26418 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:18.482326 26418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:18.482445 26418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:13:18.483657 26418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "aa6ebaa0dec848468b0897c66122bf60"
format_stamp: "Formatted at 2026-09-24 23:13:18 on dist-test-slave-m0z6"
server_key: "64de3550005c84f3bf1ca943135f82a9"
server_key_iv: "358194b7f93de918bbfe8e862d628a4d"
server_key_version: "encryptionkey@0"
I20260924 23:13:18.484011 26418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "aa6ebaa0dec848468b0897c66122bf60"
format_stamp: "Formatted at 2026-09-24 23:13:18 on dist-test-slave-m0z6"
server_key: "64de3550005c84f3bf1ca943135f82a9"
server_key_iv: "358194b7f93de918bbfe8e862d628a4d"
server_key_version: "encryptionkey@0"
I20260924 23:13:18.485525 26418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:13:18.486457 26433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:18.486783 26418 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:18.486896 26418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "aa6ebaa0dec848468b0897c66122bf60"
format_stamp: "Formatted at 2026-09-24 23:13:18 on dist-test-slave-m0z6"
server_key: "64de3550005c84f3bf1ca943135f82a9"
server_key_iv: "358194b7f93de918bbfe8e862d628a4d"
server_key_version: "encryptionkey@0"
I20260924 23:13:18.486984 26418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:18.501573 26418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:18.504038 26418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:18.504199 26418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:18.508292 26418 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:37955
I20260924 23:13:18.508458 26485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:37955 every 8 connection(s)
I20260924 23:13:18.508744 26418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:13:18.509644 26486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:18.511256 26486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:13:18.514977 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 26418
I20260924 23:13:18.515090 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:13:18.515254 13136 external_mini_cluster.cc:1428] Setting key 4ef41f7a2a76aed9953683693975a883
Sep 24 23:13:18 dist-test-slave-m0z6 krb5kdc[26405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291598, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:18.522092 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.511987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41960 (local address 127.12.212.62:37955)
0924 23:13:18.512172 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:18.512181 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:18.512293 (+ 112us) server_negotiation.cc:407] Connection header received
0924 23:13:18.512807 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:18.512809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:18.512835 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:18.512929 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:18.513541 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.514413 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.515474 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.515948 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.516079 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.518486 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:18.518495 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:18.518495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:18.518503 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:18.520346 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.520853 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.520856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.520856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.520944 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.521216 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.521217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.521218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.521458 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:18.521540 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:18.521754 (+ 214us) server_negotiation.cc:299] Negotiation successful
0924 23:13:18.521890 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":144,"thread_start_us":68,"threads_started":1}
I20260924 23:13:18.522109 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.511955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37955 (local address 127.0.0.1:41960)
0924 23:13:18.512186 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.512215 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:18.512664 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:18.512948 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:18.512951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:18.513201 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:18.513425 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:18.513429 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.514536 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.514537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:18.515387 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:18.515389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.516058 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.516059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:18.516124 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.516765 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:18.516776 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:18.518388 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:18.520450 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:18.520459 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:18.520462 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:18.520748 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:18.521011 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:18.521011 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:18.521012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:18.521129 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:18.521532 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:18.521539 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:18.521623 (+ 84us) client_negotiation.cc:769] Sending connection context
0924 23:13:18.521726 (+ 103us) client_negotiation.cc:240] Negotiation successful
0924 23:13:18.521974 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":40,"thread_start_us":87,"threads_started":1}
I20260924 23:13:18.523037 26486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:13:18.523546 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.523346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:40475 (local address 127.0.0.1:37500)
0924 23:13:18.523459 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.523513 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.61:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260924 23:13:18.523725 26434 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.212.61:40475: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.12.212.61@KRBTEST.COM; defaulting to no policy
I20260924 23:13:18.524232 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.524124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:40475 (local address 127.0.0.1:37514)
0924 23:13:18.524172 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.524206 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.61:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260924 23:13:18.524341 26486 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.61:40475: Network error: Client connection negotiation failed: client connection to 127.12.212.61:40475: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.61.keytab.
Entry for principal kudu/127.12.212.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.61.keytab.
I20260924 23:13:18.542451 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.61:40475
--webserver_interface=127.12.212.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.61.keytab
--principal=kudu/127.12.212.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:33585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:37955,127.12.212.61:40475,127.12.212.60:42285
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 23:13:18.573581 26486 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } attempt: 1
I20260924 23:13:18.574147 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.573999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:40475 (local address 127.0.0.1:37530)
0924 23:13:18.574060 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.574110 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.61:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 23:13:18.574477 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.574389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:40475 (local address 127.0.0.1:37532)
0924 23:13:18.574432 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.574458 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.61:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260924 23:13:18.574563 26486 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.61:40475: Network error: Client connection negotiation failed: client connection to 127.12.212.61:40475: connect: Connection refused (error 111)
W20260924 23:13:18.624858 26493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:18.625094 26493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:18.625119 26493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:18.626348 26493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:13:18.626395 26493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:18.626408 26493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:18.626427 26493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:13:18.626438 26493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:13:18.628190 26493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:33585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:37955,127.12.212.61:40475,127.12.212.60:42285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.61.keytab
--principal=kudu/127.12.212.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.61:40475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.212.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:18.628446 26493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:18.628631 26493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:18.633524 26498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:18.633535 26499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:18.633591 26501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:18.633886 26493 server_base.cc:1034] running on GCE node
I20260924 23:13:18.634150 26493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:18.634443 26493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:18.635643 26493 hybrid_clock.cc:648] HybridClock initialized: now 1790291598635624 us; error 31 us; skew 500 ppm
Sep 24 23:13:18 dist-test-slave-m0z6 krb5kdc[26405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291598, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:18.638286 26493 init.cc:377] Logged in from keytab as kudu/127.12.212.61@KRBTEST.COM (short username kudu)
I20260924 23:13:18.638818 26493 webserver.cc:460] Webserver started at http://127.12.212.61:35309/ using document root <none> and password file <none>
I20260924 23:13:18.639072 26493 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:18.639125 26493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:18.639249 26493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:13:18.640353 26493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-1/data/instance:
uuid: "cd4394466e854cc8b43ea22bce3e28b1"
format_stamp: "Formatted at 2026-09-24 23:13:18 on dist-test-slave-m0z6"
server_key: "0af822c88bb59724d7e23a32c0c992c3"
server_key_iv: "23f82c32b8613bbb6d7ea5f5cda9897e"
server_key_version: "encryptionkey@0"
I20260924 23:13:18.640709 26493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-1/wal/instance:
uuid: "cd4394466e854cc8b43ea22bce3e28b1"
format_stamp: "Formatted at 2026-09-24 23:13:18 on dist-test-slave-m0z6"
server_key: "0af822c88bb59724d7e23a32c0c992c3"
server_key_iv: "23f82c32b8613bbb6d7ea5f5cda9897e"
server_key_version: "encryptionkey@0"
I20260924 23:13:18.642186 26493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:13:18.642751 26486 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } attempt: 2
I20260924 23:13:18.643133 26508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:18.643247 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.643064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:40475 (local address 127.0.0.1:37534)
0924 23:13:18.643154 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.643197 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.61:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260924 23:13:18.643399 26493 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:18.643491 26493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-1/wal
uuid: "cd4394466e854cc8b43ea22bce3e28b1"
format_stamp: "Formatted at 2026-09-24 23:13:18 on dist-test-slave-m0z6"
server_key: "0af822c88bb59724d7e23a32c0c992c3"
server_key_iv: "23f82c32b8613bbb6d7ea5f5cda9897e"
server_key_version: "encryptionkey@0"
I20260924 23:13:18.643584 26493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:18.643728 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.643618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:40475 (local address 127.0.0.1:37544)
0924 23:13:18.643656 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.643693 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.61:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260924 23:13:18.643811 26486 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.61:40475: Network error: Client connection negotiation failed: client connection to 127.12.212.61:40475: connect: Connection refused (error 111)
I20260924 23:13:18.655190 26493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:18.657584 26493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:18.657730 26493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:18.661868 26493 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.61:40475
I20260924 23:13:18.661954 26560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.61:40475 every 8 connection(s)
I20260924 23:13:18.662331 26493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-1/data/info.pb
I20260924 23:13:18.663440 26561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:18.665076 26561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:13:18.667310 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 26493
I20260924 23:13:18.667400 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-1/wal/instance
I20260924 23:13:18.667563 13136 external_mini_cluster.cc:1428] Setting key 20d208e2a19fbd0efdc81018eae3b8e9
Sep 24 23:13:18 dist-test-slave-m0z6 krb5kdc[26405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291598, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.61@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.212.60@KRBTEST.COM; defaulting to no policy
I20260924 23:13:18.676371 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.665959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41976 (local address 127.12.212.62:37955)
0924 23:13:18.666069 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:18.666071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:18.666290 (+ 219us) server_negotiation.cc:407] Connection header received
0924 23:13:18.666872 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:18.666874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:18.666899 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:18.666944 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:18.667658 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.668370 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.670032 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.670267 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.670356 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.673015 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:18.673017 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:18.673018 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:18.673026 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:18.674541 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.675117 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.675120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.675120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.675180 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.675590 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.675592 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.675592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.675729 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:18.675788 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:18.676178 (+ 390us) server_negotiation.cc:299] Negotiation successful
0924 23:13:18.676272 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260924 23:13:18.676465 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.665899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37955 (local address 127.0.0.1:41976)
0924 23:13:18.666155 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.666197 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:18.666769 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:18.667027 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:18.667031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:18.667306 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:18.667555 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:18.667560 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.668651 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.668653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:18.669918 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:18.669923 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.670382 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.670384 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:18.670523 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.671085 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:18.671095 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:18.672887 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:18.674666 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:18.674679 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:18.674682 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:18.675032 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:18.675268 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:18.675270 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:18.675271 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:18.675463 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:18.675816 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:18.675824 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:18.676013 (+ 189us) client_negotiation.cc:769] Sending connection context
0924 23:13:18.676112 (+ 99us) client_negotiation.cc:240] Negotiation successful
0924 23:13:18.676272 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":120,"threads_started":1}
I20260924 23:13:18.677069 26561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.60@KRBTEST.COM" created.
Sep 24 23:13:18 dist-test-slave-m0z6 krb5kdc[26405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291598, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.61@KRBTEST.COM for kudu/127.12.212.61@KRBTEST.COM
I20260924 23:13:18.687064 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.677361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:40475 (local address 127.0.0.1:37550)
0924 23:13:18.677455 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.677469 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:18.677526 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:18.677905 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:18.677906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:18.678075 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:18.678141 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:18.678144 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.679491 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.679493 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:18.680037 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:18.680039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.680785 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.680786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:18.680886 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.681391 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:18.681397 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:18.682978 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:18.685637 (+ 2659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:18.685639 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:18.685639 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:18.685830 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:18.686164 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:18.686165 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:18.686166 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:18.686238 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:18.686671 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:18.686672 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:18.686803 (+ 131us) client_negotiation.cc:769] Sending connection context
0924 23:13:18.686877 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 23:13:18.686979 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260924 23:13:18.687192 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.677529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37550 (local address 127.12.212.61:40475)
0924 23:13:18.677747 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:18.677751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:18.677775 (+ 24us) server_negotiation.cc:407] Connection header received
0924 23:13:18.677796 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:18.677798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:18.677810 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:18.677899 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:18.678265 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.679370 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.680140 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.680652 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.680752 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.683100 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:18.683119 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:18.683119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:18.683129 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:18.685469 (+ 2340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.685949 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.685956 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.685956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.686037 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.686313 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.686314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.686314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.686565 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:18.686679 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:18.686954 (+ 275us) server_negotiation.cc:299] Negotiation successful
0924 23:13:18.687045 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":121,"threads_started":1}
I20260924 23:13:18.687968 26561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:13:18.688457 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.688258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:42285 (local address 127.0.0.1:51334)
0924 23:13:18.688370 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.688427 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:42285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260924 23:13:18.688692 26510 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.212.60:42285: connect: Connection refused (error 111)
I20260924 23:13:18.689388 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.689208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:42285 (local address 127.0.0.1:51350)
0924 23:13:18.689334 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.689362 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:42285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260924 23:13:18.689548 26561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.60:42285: Network error: Client connection negotiation failed: client connection to 127.12.212.60:42285: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.60.keytab.
Entry for principal kudu/127.12.212.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.60.keytab.
I20260924 23:13:18.698614 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.60:42285
--webserver_interface=127.12.212.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.60.keytab
--principal=kudu/127.12.212.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:33585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:37955,127.12.212.61:40475,127.12.212.60:42285
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 23:13:18.734995 26486 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } attempt: 3
I20260924 23:13:18.738821 26561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } attempt: 1
I20260924 23:13:18.739288 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.739131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:42285 (local address 127.0.0.1:51366)
0924 23:13:18.739201 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.739246 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:42285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260924 23:13:18.739709 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.739612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:42285 (local address 127.0.0.1:51380)
0924 23:13:18.739651 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.739679 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:42285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 23:13:18.739830 26561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.60:42285: Network error: Client connection negotiation failed: client connection to 127.12.212.60:42285: connect: Connection refused (error 111)
Sep 24 23:13:18 dist-test-slave-m0z6 krb5kdc[26405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291598, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for kudu/127.12.212.61@KRBTEST.COM
I20260924 23:13:18.743533 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.735369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:40475 (local address 127.0.0.1:37566)
0924 23:13:18.735433 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.735447 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:18.735499 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:18.735724 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:18.735726 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:18.735894 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:18.736043 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:18.736047 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.737038 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.737040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:18.737583 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:18.737585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.738176 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.738177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:18.738256 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.738876 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:18.738882 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:18.740384 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:18.741989 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:18.741991 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:18.741991 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:18.742227 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:18.742676 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:18.742677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:18.742678 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:18.742809 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:18.743163 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:18.743164 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:18.743227 (+ 63us) client_negotiation.cc:769] Sending connection context
0924 23:13:18.743308 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 23:13:18.743430 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260924 23:13:18.743585 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.735475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37566 (local address 127.12.212.61:40475)
0924 23:13:18.735542 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:18.735544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:18.735556 (+ 12us) server_negotiation.cc:407] Connection header received
0924 23:13:18.735587 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:18.735589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:18.735602 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:18.735685 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:18.736139 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.736936 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.737671 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.738076 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.738152 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.740505 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:18.740507 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:18.740507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:18.740514 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:18.741896 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.742485 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.742486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.742487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.742601 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.742946 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.742948 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.742948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.743088 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:18.743159 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:18.743429 (+ 270us) server_negotiation.cc:299] Negotiation successful
0924 23:13:18.743519 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 23:13:18.743968 26486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:13:18.744335 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.744228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:42285 (local address 127.0.0.1:51382)
0924 23:13:18.744275 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.744309 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:42285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 23:13:18.744674 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.744598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:42285 (local address 127.0.0.1:51384)
0924 23:13:18.744631 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.744656 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:42285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260924 23:13:18.744751 26486 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.60:42285: Network error: Client connection negotiation failed: client connection to 127.12.212.60:42285: connect: Connection refused (error 111)
I20260924 23:13:18.775988 26486 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } attempt: 1
I20260924 23:13:18.776585 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.776424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:42285 (local address 127.0.0.1:51398)
0924 23:13:18.776498 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.776548 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:42285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260924 23:13:18.777005 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.776904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:42285 (local address 127.0.0.1:51414)
0924 23:13:18.776950 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.776981 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:42285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 23:13:18.777184 26486 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.60:42285: Network error: Client connection negotiation failed: client connection to 127.12.212.60:42285: connect: Connection refused (error 111)
W20260924 23:13:18.784403 26568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:18.784636 26568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:18.784660 26568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:18.786275 26568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:13:18.786356 26568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:18.786373 26568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:18.786386 26568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:13:18.786402 26568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:13:18.788194 26568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:33585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:37955,127.12.212.61:40475,127.12.212.60:42285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.60.keytab
--principal=kudu/127.12.212.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.60:42285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.212.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:18.788467 26568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:18.788657 26568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:18.793881 26576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:18.793856 26573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:18.793856 26574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:18.793861 26568 server_base.cc:1034] running on GCE node
I20260924 23:13:18.794420 26568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:18.794782 26568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:18.795977 26568 hybrid_clock.cc:648] HybridClock initialized: now 1790291598795956 us; error 37 us; skew 500 ppm
Sep 24 23:13:18 dist-test-slave-m0z6 krb5kdc[26405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291598, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:18.798593 26568 init.cc:377] Logged in from keytab as kudu/127.12.212.60@KRBTEST.COM (short username kudu)
I20260924 23:13:18.799193 26568 webserver.cc:460] Webserver started at http://127.12.212.60:34203/ using document root <none> and password file <none>
I20260924 23:13:18.799396 26568 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:18.799438 26568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:18.799527 26568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:13:18.800587 26568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-2/data/instance:
uuid: "badad613de6145d3b9c8eabbdf48a2b9"
format_stamp: "Formatted at 2026-09-24 23:13:18 on dist-test-slave-m0z6"
server_key: "2a732e98602da7bfe33934ff21c6dfd2"
server_key_iv: "bfb3002292fa55f3a88d371371d1323d"
server_key_version: "encryptionkey@0"
I20260924 23:13:18.800917 26568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-2/wal/instance:
uuid: "badad613de6145d3b9c8eabbdf48a2b9"
format_stamp: "Formatted at 2026-09-24 23:13:18 on dist-test-slave-m0z6"
server_key: "2a732e98602da7bfe33934ff21c6dfd2"
server_key_iv: "bfb3002292fa55f3a88d371371d1323d"
server_key_version: "encryptionkey@0"
I20260924 23:13:18.802310 26568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:13:18.803231 26583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:18.803450 26568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 23:13:18.803555 26568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-2/wal
uuid: "badad613de6145d3b9c8eabbdf48a2b9"
format_stamp: "Formatted at 2026-09-24 23:13:18 on dist-test-slave-m0z6"
server_key: "2a732e98602da7bfe33934ff21c6dfd2"
server_key_iv: "bfb3002292fa55f3a88d371371d1323d"
server_key_version: "encryptionkey@0"
I20260924 23:13:18.803620 26568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:18.807993 26561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } attempt: 2
I20260924 23:13:18.808485 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.808338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:42285 (local address 127.0.0.1:51424)
0924 23:13:18.808401 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.808452 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:42285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260924 23:13:18.808910 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.808794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:42285 (local address 127.0.0.1:51426)
0924 23:13:18.808843 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.808875 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:42285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260924 23:13:18.809028 26561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.60:42285: Network error: Client connection negotiation failed: client connection to 127.12.212.60:42285: connect: Connection refused (error 111)
I20260924 23:13:18.814792 26568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:18.816982 26568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:18.817104 26568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:18.820889 26568 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.60:42285
I20260924 23:13:18.820935 26635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.60:42285 every 8 connection(s)
I20260924 23:13:18.821262 26568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-2/data/info.pb
I20260924 23:13:18.822134 26636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:18.823868 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 26568
I20260924 23:13:18.823976 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790291444018253-13136-0/minicluster-data/master-2/wal/instance
I20260924 23:13:18.823874 26636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:13:18.824156 13136 external_mini_cluster.cc:1428] Setting key 005904b24a078d95c9131ed50becf5f8
Sep 24 23:13:18 dist-test-slave-m0z6 krb5kdc[26405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291598, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.60@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
Sep 24 23:13:18 dist-test-slave-m0z6 krb5kdc[26405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.60@KRBTEST.COM
I20260924 23:13:18.834285 26639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.824666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:37955 (local address 127.0.0.1:41986)
0924 23:13:18.824969 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.825006 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:18.825456 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:18.825711 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:18.825713 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:18.825989 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:18.826159 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:18.826164 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.826911 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.826913 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:18.827822 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:18.827824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.828345 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.828347 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:18.828508 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.829050 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:18.829061 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:18.830802 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:18.832557 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:18.832568 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:18.832571 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:18.832996 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:18.833193 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:18.833194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:18.833194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:18.833334 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:18.833574 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:18.833579 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:18.833759 (+ 180us) client_negotiation.cc:769] Sending connection context
0924 23:13:18.833875 (+ 116us) client_negotiation.cc:240] Negotiation successful
0924 23:13:18.834027 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"spinlock_wait_cycles":384,"thread_start_us":136,"threads_started":1}
I20260924 23:13:18.834467 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.824679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41986 (local address 127.12.212.62:37955)
0924 23:13:18.824734 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:18.824736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:18.825108 (+ 372us) server_negotiation.cc:407] Connection header received
0924 23:13:18.825595 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:18.825597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:18.825609 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:18.825689 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:18.826251 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.826838 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.827922 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.828221 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.828312 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.830918 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:18.830920 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:18.830921 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:18.830930 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:18.832463 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.833072 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.833074 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.833074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.833133 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.833400 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.833400 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.833400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.833516 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:18.833565 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:18.834288 (+ 723us) server_negotiation.cc:299] Negotiation successful
0924 23:13:18.834366 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 23:13:18.835116 26636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:13:18.835224 26640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.824666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51440 (local address 127.12.212.60:42285)
0924 23:13:18.824876 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:18.824887 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:18.824926 (+ 39us) server_negotiation.cc:407] Connection header received
0924 23:13:18.825443 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:18.825455 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:18.825482 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:18.825581 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:18.826260 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.827323 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.828844 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.829277 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.829379 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.831936 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:18.831947 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:18.831948 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:18.831955 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:18.833804 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.834257 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.834260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.834260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.834322 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.834552 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.834552 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.834553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.834756 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:18.834909 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:18.835023 (+ 114us) server_negotiation.cc:299] Negotiation successful
0924 23:13:18.835105 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":100,"threads_started":1}
Sep 24 23:13:18 dist-test-slave-m0z6 krb5kdc[26405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291598, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.60@KRBTEST.COM for kudu/127.12.212.61@KRBTEST.COM
Sep 24 23:13:18 dist-test-slave-m0z6 krb5kdc[26405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:18.842653 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.835471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37582 (local address 127.12.212.61:40475)
0924 23:13:18.835540 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:18.835543 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:18.835551 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:13:18.835577 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:18.835578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:18.835588 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:18.835643 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:18.836019 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.836587 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.837312 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.837498 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.837548 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.839783 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:18.839785 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:18.839786 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:18.839795 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:18.841329 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.841733 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.841735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.841735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.841810 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.842123 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.842124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.842125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.842263 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:18.842339 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:18.842510 (+ 171us) server_negotiation.cc:299] Negotiation successful
0924 23:13:18.842582 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 23:13:18.842687 26639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.835395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:40475 (local address 127.0.0.1:37582)
0924 23:13:18.835435 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.835448 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:18.835487 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:18.835658 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:18.835659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:18.835807 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:18.835939 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:18.835942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.836662 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.836663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:18.837227 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:18.837229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.837561 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.837562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:18.837639 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.838197 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:18.838202 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:18.839650 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:18.841426 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:18.841428 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:18.841428 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:18.841642 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:18.841954 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:18.841956 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:18.841956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:18.842031 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:18.842337 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:18.842339 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:18.842397 (+ 58us) client_negotiation.cc:769] Sending connection context
0924 23:13:18.842472 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 23:13:18.842587 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260924 23:13:18.843163 26636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:13:18.843883 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.836130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42002 (local address 127.12.212.62:37955)
0924 23:13:18.836172 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:18.836173 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:18.836181 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:13:18.836202 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:18.836202 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:18.836211 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:18.836289 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:18.836704 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.837138 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.837983 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.838252 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.838308 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.840811 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:18.840813 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:18.840813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:18.840823 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:18.842422 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.842848 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.842849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.842849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.842906 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.843083 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.843085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.843085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.843409 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:18.843623 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:18.843659 (+ 36us) server_negotiation.cc:299] Negotiation successful
0924 23:13:18.843738 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Sep 24 23:13:18 dist-test-slave-m0z6 krb5kdc[26405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291598, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.60@KRBTEST.COM for kudu/127.12.212.60@KRBTEST.COM
Sep 24 23:13:18 dist-test-slave-m0z6 krb5kdc[26405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.61@KRBTEST.COM
I20260924 23:13:18.850195 26640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.843479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51456 (local address 127.12.212.60:42285)
0924 23:13:18.843810 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:18.843812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:18.843820 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:13:18.843840 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:18.843840 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:18.843851 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:18.843908 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:18.844230 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.844844 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.845659 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.845864 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.845915 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.847830 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:18.847831 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:18.847831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:18.847839 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:18.849139 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.849462 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.849463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.849464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.849514 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.849694 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.849695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.849695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.849806 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:18.849858 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:18.849989 (+ 131us) server_negotiation.cc:299] Negotiation successful
0924 23:13:18.850081 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":309}
I20260924 23:13:18.850238 26639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.843445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:42285 (local address 127.0.0.1:51456)
0924 23:13:18.843477 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.843487 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:18.843522 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:18.843947 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:18.843948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:18.844110 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:18.844147 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:18.844149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.845022 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.845023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:18.845579 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:18.845582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.845924 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.845925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:18.845980 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.846432 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:18.846437 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:18.847757 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:18.849230 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:18.849231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:18.849231 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:18.849394 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:18.849566 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:18.849567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:18.849567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:18.849644 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:18.849856 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:18.849856 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:18.849904 (+ 48us) client_negotiation.cc:769] Sending connection context
0924 23:13:18.849963 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 23:13:18.850128 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15,"mutex_wait_us":73}
I20260924 23:13:18.852172 26636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9: Bootstrap starting.
I20260924 23:13:18.852319 26486 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } attempt: 2
I20260924 23:13:18.852773 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.844956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37592 (local address 127.12.212.61:40475)
0924 23:13:18.845014 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:18.845016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:18.845023 (+ 7us) server_negotiation.cc:407] Connection header received
0924 23:13:18.845043 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:18.845044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:18.845053 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:18.845098 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:18.845541 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.846096 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.846933 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.847126 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.847188 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.849598 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:18.849600 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:18.849600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:18.849608 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:18.851070 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.851470 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.851471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.851472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.851533 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.851754 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.851755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.851755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.851893 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:18.852535 (+ 642us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:18.852584 (+ 49us) server_negotiation.cc:299] Negotiation successful
0924 23:13:18.852669 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 23:13:18.853257 26636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9: Neither blocks nor log segments found. Creating new log.
I20260924 23:13:18.853564 26636 log.cc:824] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:18.854458 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:13:18.854542 26636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9: No bootstrap required, opened a new log
I20260924 23:13:18.856562 26636 raft_consensus.cc:348] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } } peers { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } } peers { permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } }
I20260924 23:13:18.856805 26636 raft_consensus.cc:374] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:13:18.856850 26636 raft_consensus.cc:729] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: badad613de6145d3b9c8eabbdf48a2b9, State: Initialized, Role: FOLLOWER
I20260924 23:13:18.857016 26636 consensus_queue.cc:260] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } } peers { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } } peers { permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } }
I20260924 23:13:18.857347 26636 sys_catalog.cc:564] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9 [sys.catalog]: configured and running, proceeding with master startup.
Sep 24 23:13:18 dist-test-slave-m0z6 krb5kdc[26405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291598, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for kudu/127.12.212.60@KRBTEST.COM
W20260924 23:13:18.859292 26659 catalog_manager.cc:1558] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:13:18.859403 26659 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:13:18.860589 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.852564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:42285 (local address 127.0.0.1:51472)
0924 23:13:18.852612 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.852622 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:18.852656 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:18.853035 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:18.853037 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:18.853211 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:18.853262 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:18.853264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.854186 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.854188 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:18.854955 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:18.854959 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.855426 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.855427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:18.855493 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.856282 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:18.856288 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:18.857827 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:18.859557 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:18.859558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:18.859558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:18.859764 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:18.859963 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:18.859963 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:18.859964 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:18.860020 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:18.860293 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:18.860294 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:18.860351 (+ 57us) client_negotiation.cc:769] Sending connection context
0924 23:13:18.860395 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 23:13:18.860489 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 23:13:18.860934 26640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.852589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51472 (local address 127.12.212.60:42285)
0924 23:13:18.852886 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:18.852888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:18.852897 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:13:18.852918 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:18.852919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:18.852929 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:18.853015 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:18.853380 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.854117 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.855054 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.855338 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.856046 (+ 708us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.857914 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:18.857915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:18.857916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:18.857923 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:18.859478 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.859843 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.859844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.859844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.859899 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.860078 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.860079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.860079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.860223 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:18.860708 (+ 485us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:18.860773 (+ 65us) server_negotiation.cc:299] Negotiation successful
0924 23:13:18.860855 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275}
I20260924 23:13:18.861153 26648 sys_catalog.cc:455] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } } peers { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } } peers { permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } } }
I20260924 23:13:18.861243 26648 sys_catalog.cc:458] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:13:18.862833 26486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60: Bootstrap starting.
I20260924 23:13:18.863749 26486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60: Neither blocks nor log segments found. Creating new log.
I20260924 23:13:18.864099 26486 log.cc:824] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:18.865389 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.856183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37594 (local address 127.12.212.61:40475)
0924 23:13:18.856227 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:18.856229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:18.856565 (+ 336us) server_negotiation.cc:407] Connection header received
0924 23:13:18.856782 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:18.856783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:18.856795 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:18.856826 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:18.857944 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.858631 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.860002 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.860319 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.860363 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.862318 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:18.862319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:18.862320 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:18.862328 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:18.864058 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.864516 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.864518 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.864518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.864583 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.864795 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.864797 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.864797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.864959 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:18.865079 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:18.865211 (+ 132us) server_negotiation.cc:299] Negotiation successful
0924 23:13:18.865299 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 23:13:18.859688 26568 master_runner.cc:186] Error getting master registration for 127.12.212.62:37955: OK, instance_id { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" instance_seqno: 1790291598507080 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 23:13:18.866266 26568 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 23:13:18.866990 26486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60: No bootstrap required, opened a new log
I20260924 23:13:18.871315 26486 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } } peers { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } } peers { permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } }
I20260924 23:13:18.871495 26486 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:13:18.871546 26486 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa6ebaa0dec848468b0897c66122bf60, State: Initialized, Role: FOLLOWER
I20260924 23:13:18.871687 26486 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } } peers { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } } peers { permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } }
I20260924 23:13:18.872128 26486 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:13:18.872524 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.856282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42014 (local address 127.12.212.62:37955)
0924 23:13:18.856322 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:18.856324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:18.856332 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:13:18.856351 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:18.856352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:18.856361 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:18.856430 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:18.857120 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.857820 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.867157 (+ 9337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.867387 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.867455 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.869110 (+ 1655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:18.869112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:18.869112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:18.869120 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:18.870621 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.871135 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.871137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.871137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.871208 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.871427 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.871428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.871428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.871851 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:18.871943 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:18.872349 (+ 406us) server_negotiation.cc:299] Negotiation successful
0924 23:13:18.872430 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 23:13:18.873287 26663 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } } peers { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } } peers { permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } } }
I20260924 23:13:18.873376 26663 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [sys.catalog]: This master's current role is: FOLLOWER
W20260924 23:13:18.874224 26674 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:13:18.874276 26674 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:13:18.874472 26640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.861500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51484 (local address 127.12.212.60:42285)
0924 23:13:18.861534 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:18.861536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:18.865505 (+ 3969us) server_negotiation.cc:407] Connection header received
0924 23:13:18.865604 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:18.865605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:18.865617 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:18.865645 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:18.866306 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.867065 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.868394 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.868723 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.868769 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.870294 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:18.870295 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:18.870296 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:18.870304 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:18.872378 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.873202 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.873204 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.873204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.873276 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.873592 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.873593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.873594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.873739 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:18.873824 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:18.874300 (+ 476us) server_negotiation.cc:299] Negotiation successful
0924 23:13:18.874384 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 23:13:18.875005 26418 master_runner.cc:186] Error getting master registration for 127.12.212.61:40475: OK, instance_id { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" instance_seqno: 1790291598660480 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 23:13:18.877077 26418 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 23:13:18.900214 26561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } attempt: 3
Sep 24 23:13:18 dist-test-slave-m0z6 krb5kdc[26405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291598, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.61@KRBTEST.COM for kudu/127.12.212.60@KRBTEST.COM
I20260924 23:13:18.907212 26640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.900661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51488 (local address 127.12.212.60:42285)
0924 23:13:18.900709 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:18.900711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:18.900720 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:13:18.900775 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:18.900775 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:18.900786 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:18.900829 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:18.901179 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.901646 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.902387 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.902578 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.902640 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.904709 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:18.904711 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:18.904711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:18.904719 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:18.906017 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.906361 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.906362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.906363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.906419 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:18.906659 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:18.906659 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:18.906659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:18.906822 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:18.906883 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:18.907033 (+ 150us) server_negotiation.cc:299] Negotiation successful
0924 23:13:18.907116 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 23:13:18.907240 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:18.900583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:42285 (local address 127.0.0.1:51488)
0924 23:13:18.900645 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:18.900659 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:18.900709 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:18.900884 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:18.900884 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:18.901047 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:18.901093 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:18.901096 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.901722 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.901723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:18.902224 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:18.902226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:18.902653 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:18.902654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:18.902751 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:18.903181 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:18.903186 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:18.904607 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:18.906091 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:18.906092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:18.906093 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:18.906297 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:18.906474 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:18.906475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:18.906475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:18.906535 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:18.906883 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:18.906884 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:18.906944 (+ 60us) client_negotiation.cc:769] Sending connection context
0924 23:13:18.907005 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 23:13:18.907155 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260924 23:13:18.908674 26561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1: Bootstrap starting.
I20260924 23:13:18.909322 26561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1: Neither blocks nor log segments found. Creating new log.
I20260924 23:13:18.909623 26561 log.cc:824] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:18.910426 26561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1: No bootstrap required, opened a new log
I20260924 23:13:18.912024 26561 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } } peers { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } } peers { permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } }
I20260924 23:13:18.912549 26561 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:13:18.912603 26561 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4394466e854cc8b43ea22bce3e28b1, State: Initialized, Role: FOLLOWER
I20260924 23:13:18.912740 26561 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } } peers { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } } peers { permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } }
I20260924 23:13:18.913020 26677 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } } peers { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } } peers { permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } } }
I20260924 23:13:18.913100 26561 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:13:18.913160 26677 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1 [sys.catalog]: This master's current role is: FOLLOWER
W20260924 23:13:18.914839 26688 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:13:18.914951 26688 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:13:18.917193 26493 master_runner.cc:263] No leader master found from master cd4394466e854cc8b43ea22bce3e28b1
I20260924 23:13:18.917244 26493 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 23:13:19.005162 26439 tablet.cc:2369] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 23:13:19.067787 26589 tablet.cc:2369] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:13:19.125572 26663 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:13:19.125715 26663 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } } peers { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } } peers { permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } }
I20260924 23:13:19.125985 26663 leader_election.cc:288] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd4394466e854cc8b43ea22bce3e28b1 (127.12.212.61:40475), badad613de6145d3b9c8eabbdf48a2b9 (127.12.212.60:42285)
I20260924 23:13:19.126368 26610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aa6ebaa0dec848468b0897c66122bf60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "badad613de6145d3b9c8eabbdf48a2b9" is_pre_election: true
I20260924 23:13:19.126536 26610 raft_consensus.cc:2454] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa6ebaa0dec848468b0897c66122bf60 in term 0.
I20260924 23:13:19.126791 26535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aa6ebaa0dec848468b0897c66122bf60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4394466e854cc8b43ea22bce3e28b1" is_pre_election: true
I20260924 23:13:19.126816 26435 leader_election.cc:302] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa6ebaa0dec848468b0897c66122bf60, badad613de6145d3b9c8eabbdf48a2b9; no voters:
I20260924 23:13:19.126924 26535 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa6ebaa0dec848468b0897c66122bf60 in term 0.
I20260924 23:13:19.127069 26663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:13:19.127152 26663 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:13:19.127177 26663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:13:19.127874 26663 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } } peers { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } } peers { permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } }
I20260924 23:13:19.128000 26663 leader_election.cc:288] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [CANDIDATE]: Term 1 election: Requested vote from peers cd4394466e854cc8b43ea22bce3e28b1 (127.12.212.61:40475), badad613de6145d3b9c8eabbdf48a2b9 (127.12.212.60:42285)
I20260924 23:13:19.128192 26535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aa6ebaa0dec848468b0897c66122bf60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4394466e854cc8b43ea22bce3e28b1"
I20260924 23:13:19.128285 26535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:13:19.128996 26535 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa6ebaa0dec848468b0897c66122bf60 in term 1.
I20260924 23:13:19.129115 26610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aa6ebaa0dec848468b0897c66122bf60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "badad613de6145d3b9c8eabbdf48a2b9"
I20260924 23:13:19.129182 26434 leader_election.cc:302] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa6ebaa0dec848468b0897c66122bf60, cd4394466e854cc8b43ea22bce3e28b1; no voters:
I20260924 23:13:19.129213 26610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:13:19.129271 26663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:13:19.129405 26663 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [term 1 LEADER]: Becoming Leader. State: Replica: aa6ebaa0dec848468b0897c66122bf60, State: Running, Role: LEADER
I20260924 23:13:19.129504 26663 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } } peers { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } } peers { permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } }
I20260924 23:13:19.129944 26690 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa6ebaa0dec848468b0897c66122bf60. Latest consensus state: current_term: 1 leader_uuid: "aa6ebaa0dec848468b0897c66122bf60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } } peers { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } } peers { permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } } }
I20260924 23:13:19.130033 26690 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [sys.catalog]: This master's current role is: LEADER
I20260924 23:13:19.130223 26692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:13:19.130234 26610 raft_consensus.cc:2454] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa6ebaa0dec848468b0897c66122bf60 in term 1.
I20260924 23:13:19.130405 26692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:13:19.131829 26610 raft_consensus.cc:1261] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9 [term 1 FOLLOWER]: Refusing update from remote peer aa6ebaa0dec848468b0897c66122bf60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:13:19.131846 26535 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1 [term 1 FOLLOWER]: Refusing update from remote peer aa6ebaa0dec848468b0897c66122bf60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:13:19.132166 26690 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:13:19.132287 26663 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:13:19.133446 26648 sys_catalog.cc:455] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa6ebaa0dec848468b0897c66122bf60. Latest consensus state: current_term: 1 leader_uuid: "aa6ebaa0dec848468b0897c66122bf60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } } peers { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } } peers { permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } } }
I20260924 23:13:19.133448 26677 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa6ebaa0dec848468b0897c66122bf60. Latest consensus state: current_term: 1 leader_uuid: "aa6ebaa0dec848468b0897c66122bf60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } } peers { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } } peers { permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } } }
I20260924 23:13:19.133533 26648 sys_catalog.cc:458] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:13:19.133533 26677 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:13:19.134418 26690 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aa6ebaa0dec848468b0897c66122bf60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } } peers { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } } peers { permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } } }
I20260924 23:13:19.134509 26690 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [sys.catalog]: This master's current role is: LEADER
I20260924 23:13:19.134610 26648 sys_catalog.cc:455] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa6ebaa0dec848468b0897c66122bf60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } } peers { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } } peers { permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } } }
I20260924 23:13:19.134747 26648 sys_catalog.cc:458] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:13:19.134744 26677 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa6ebaa0dec848468b0897c66122bf60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } } peers { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } } peers { permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } } }
I20260924 23:13:19.134832 26677 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:13:19.134559 26690 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aa6ebaa0dec848468b0897c66122bf60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6ebaa0dec848468b0897c66122bf60" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 37955 } } peers { permanent_uuid: "cd4394466e854cc8b43ea22bce3e28b1" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 40475 } } peers { permanent_uuid: "badad613de6145d3b9c8eabbdf48a2b9" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 42285 } } }
I20260924 23:13:19.134857 26690 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60 [sys.catalog]: This master's current role is: LEADER
I20260924 23:13:19.134910 26695 mvcc.cc:205] Tried to move back new op lower bound from 7333034390041706496 to 7333034390035091456. Current Snapshot: MvccSnapshot[applied={T|T < 7333034390041706496}]
I20260924 23:13:19.135102 26694 mvcc.cc:205] Tried to move back new op lower bound from 7333034390041706496 to 7333034390035091456. Current Snapshot: MvccSnapshot[applied={T|T < 7333034390041706496}]
I20260924 23:13:19.135607 26692 catalog_manager.cc:1347] Generated new cluster ID: fb5cff82d74143bd826642aee5617697
I20260924 23:13:19.135663 26692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:13:19.161258 26692 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:13:19.161947 26692 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:13:19.170609 26692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa6ebaa0dec848468b0897c66122bf60: Generated new TSK 0
I20260924 23:13:19.170903 26692 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:13:19.859826 26659 catalog_manager.cc:1259] Loaded cluster ID: fb5cff82d74143bd826642aee5617697
I20260924 23:13:19.859910 26659 catalog_manager.cc:1552] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9: loading cluster ID for follower catalog manager: success
I20260924 23:13:19.860767 26659 catalog_manager.cc:1574] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9: acquiring CA information for follower catalog manager: success
I20260924 23:13:19.861078 26659 catalog_manager.cc:1602] T 00000000000000000000000000000000 P badad613de6145d3b9c8eabbdf48a2b9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 23:13:19.915473 26688 catalog_manager.cc:1259] Loaded cluster ID: fb5cff82d74143bd826642aee5617697
I20260924 23:13:19.915560 26688 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1: loading cluster ID for follower catalog manager: success
I20260924 23:13:19.916391 26688 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1: acquiring CA information for follower catalog manager: success
I20260924 23:13:19.916705 26688 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cd4394466e854cc8b43ea22bce3e28b1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 23:13:20.233559 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 26418
I20260924 23:13:20.238337 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 26493
I20260924 23:13:20.242607 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 26568
2026-09-24T23:13:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2953 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:13:20 dist-test-slave-m0z6 krb5kdc[26707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:13:20 dist-test-slave-m0z6 krb5kdc[26707](info): set up 2 sockets
Sep 24 23:13:20 dist-test-slave-m0z6 krb5kdc[26707](info): commencing operation
krb5kdc: starting...
W20260924 23:13:21.202586 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:13:21 dist-test-slave-m0z6 krb5kdc[26707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:13:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:13:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:13:21.321887 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:43155
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:36283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:43155
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:13:21.404193 26720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:21.404422 26720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:21.404445 26720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:21.405686 26720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:13:21.405737 26720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:21.405751 26720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:21.405763 26720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:13:21.405776 26720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:13:21.407397 26720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:36283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:43155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:43155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:21.407678 26720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:21.407915 26720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:21.412832 26726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:21.412842 26728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:21.412972 26725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:21.413002 26720 server_base.cc:1034] running on GCE node
I20260924 23:13:21.413282 26720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:21.413619 26720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:21.414784 26720 hybrid_clock.cc:648] HybridClock initialized: now 1790291601414762 us; error 38 us; skew 500 ppm
Sep 24 23:13:21 dist-test-slave-m0z6 krb5kdc[26707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291601, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:21.417284 26720 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:13:21.417773 26720 webserver.cc:460] Webserver started at http://127.12.212.62:43313/ using document root <none> and password file <none>
I20260924 23:13:21.417979 26720 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:21.418027 26720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:21.418160 26720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:13:21.419298 26720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "b777c7d6be4540d3a58ae2e789a290a6"
format_stamp: "Formatted at 2026-09-24 23:13:21 on dist-test-slave-m0z6"
server_key: "fc71b5e64b3f2b924f389e54e50b861a"
server_key_iv: "9f6b15ce4f23ffe11b3521509ff1c8b7"
server_key_version: "encryptionkey@0"
I20260924 23:13:21.419655 26720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "b777c7d6be4540d3a58ae2e789a290a6"
format_stamp: "Formatted at 2026-09-24 23:13:21 on dist-test-slave-m0z6"
server_key: "fc71b5e64b3f2b924f389e54e50b861a"
server_key_iv: "9f6b15ce4f23ffe11b3521509ff1c8b7"
server_key_version: "encryptionkey@0"
I20260924 23:13:21.421276 26720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:13:21.422247 26735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:21.422461 26720 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:21.422565 26720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "b777c7d6be4540d3a58ae2e789a290a6"
format_stamp: "Formatted at 2026-09-24 23:13:21 on dist-test-slave-m0z6"
server_key: "fc71b5e64b3f2b924f389e54e50b861a"
server_key_iv: "9f6b15ce4f23ffe11b3521509ff1c8b7"
server_key_version: "encryptionkey@0"
I20260924 23:13:21.422648 26720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:21.432318 26720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:21.434214 26720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:21.434350 26720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:21.438086 26720 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:43155
I20260924 23:13:21.438120 26787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:43155 every 8 connection(s)
I20260924 23:13:21.438465 26720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:13:21.439317 26788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:21.441640 26788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6: Bootstrap starting.
I20260924 23:13:21.442237 26788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6: Neither blocks nor log segments found. Creating new log.
I20260924 23:13:21.442519 26788 log.cc:824] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:21.443454 26788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6: No bootstrap required, opened a new log
I20260924 23:13:21.445118 26788 raft_consensus.cc:348] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b777c7d6be4540d3a58ae2e789a290a6" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 43155 } }
I20260924 23:13:21.445272 26788 raft_consensus.cc:374] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:13:21.445294 26788 raft_consensus.cc:729] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b777c7d6be4540d3a58ae2e789a290a6, State: Initialized, Role: FOLLOWER
I20260924 23:13:21.445446 26788 consensus_queue.cc:260] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b777c7d6be4540d3a58ae2e789a290a6" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 43155 } }
I20260924 23:13:21.445518 26788 raft_consensus.cc:388] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:13:21.445541 26788 raft_consensus.cc:482] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:13:21.445573 26788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:13:21.446130 26788 raft_consensus.cc:504] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b777c7d6be4540d3a58ae2e789a290a6" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 43155 } }
I20260924 23:13:21.446220 26788 leader_election.cc:302] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b777c7d6be4540d3a58ae2e789a290a6; no voters:
I20260924 23:13:21.446398 26788 leader_election.cc:288] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:13:21.446460 26791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:13:21.446650 26788 sys_catalog.cc:564] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:13:21.446780 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 26720
I20260924 23:13:21.446887 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:13:21.447070 13136 external_mini_cluster.cc:1428] Setting key d65b9fcc611501b86512b47ecf21ac30
I20260924 23:13:21.447302 26791 raft_consensus.cc:686] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6 [term 1 LEADER]: Becoming Leader. State: Replica: b777c7d6be4540d3a58ae2e789a290a6, State: Running, Role: LEADER
I20260924 23:13:21.447409 26791 consensus_queue.cc:237] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b777c7d6be4540d3a58ae2e789a290a6" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 43155 } }
I20260924 23:13:21.448020 26791 sys_catalog.cc:455] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b777c7d6be4540d3a58ae2e789a290a6. Latest consensus state: current_term: 1 leader_uuid: "b777c7d6be4540d3a58ae2e789a290a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b777c7d6be4540d3a58ae2e789a290a6" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 43155 } } }
I20260924 23:13:21.448096 26791 sys_catalog.cc:458] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6 [sys.catalog]: This master's current role is: LEADER
I20260924 23:13:21.448239 26792 sys_catalog.cc:455] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b777c7d6be4540d3a58ae2e789a290a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b777c7d6be4540d3a58ae2e789a290a6" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 43155 } } }
I20260924 23:13:21.448304 26808 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:13:21.448350 26792 sys_catalog.cc:458] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6 [sys.catalog]: This master's current role is: LEADER
I20260924 23:13:21.448849 26808 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:13:21.450330 26808 catalog_manager.cc:1347] Generated new cluster ID: 5ea359c70c934ec38232c2563d8a2c29
I20260924 23:13:21.450383 26808 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 23:13:21 dist-test-slave-m0z6 krb5kdc[26707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:21.459627 26800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:21.447507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59468 (local address 127.12.212.62:43155)
0924 23:13:21.448370 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:21.448382 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:21.448417 (+ 35us) server_negotiation.cc:407] Connection header received
0924 23:13:21.448557 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:21.448565 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:21.448631 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:21.448788 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:21.449528 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:21.450929 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:21.452131 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:21.452812 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:21.452928 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:21.455713 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:21.455718 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:21.455718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:21.455732 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:21.457719 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:21.458161 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:21.458164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:21.458164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:21.458280 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:21.458475 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:21.458476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:21.458476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:21.458845 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:21.459002 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:21.459116 (+ 114us) server_negotiation.cc:299] Negotiation successful
0924 23:13:21.459372 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":78,"threads_started":1}
I20260924 23:13:21.460584 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:13:21.464257 26808 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:13:21.464797 26808 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:13:21.466755 26808 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b777c7d6be4540d3a58ae2e789a290a6: Generated new TSK 0
I20260924 23:13:21.466990 26808 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:13:21.514653 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 26720
2026-09-24T23:13:21Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1269 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:13:21 dist-test-slave-m0z6 krb5kdc[26815](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:13:21 dist-test-slave-m0z6 krb5kdc[26815](info): set up 2 sockets
Sep 24 23:13:21 dist-test-slave-m0z6 krb5kdc[26815](info): commencing operation
krb5kdc: starting...
W20260924 23:13:22.470523 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:13:22 dist-test-slave-m0z6 krb5kdc[26815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:13:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:13:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:13:22.591789 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:42995
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:42995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:13:22.674118 26828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:22.674348 26828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:22.674372 26828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:22.675740 26828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:13:22.675803 26828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:22.675817 26828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:22.675828 26828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:13:22.675839 26828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:13:22.677435 26828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:42995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:42995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:22.677695 26828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:22.677961 26828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:13:22.682989 26828 server_base.cc:1034] running on GCE node
W20260924 23:13:22.682960 26834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:22.682960 26833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:22.683019 26836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:22.683514 26828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:22.683833 26828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:22.685024 26828 hybrid_clock.cc:648] HybridClock initialized: now 1790291602685003 us; error 38 us; skew 500 ppm
Sep 24 23:13:22 dist-test-slave-m0z6 krb5kdc[26815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291602, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:22.688156 26828 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:13:22.688730 26828 webserver.cc:460] Webserver started at http://127.12.212.62:35103/ using document root <none> and password file <none>
I20260924 23:13:22.688946 26828 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:22.689006 26828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:22.689134 26828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:13:22.690287 26828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "99a72e374aa044f0afbc5774ba21a5ec"
format_stamp: "Formatted at 2026-09-24 23:13:22 on dist-test-slave-m0z6"
server_key: "ecfc3bea0cc73bcc15b9dfb5b5813a80"
server_key_iv: "c13fc02a10790ef7a8f932b0ebf618ef"
server_key_version: "encryptionkey@0"
I20260924 23:13:22.690668 26828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "99a72e374aa044f0afbc5774ba21a5ec"
format_stamp: "Formatted at 2026-09-24 23:13:22 on dist-test-slave-m0z6"
server_key: "ecfc3bea0cc73bcc15b9dfb5b5813a80"
server_key_iv: "c13fc02a10790ef7a8f932b0ebf618ef"
server_key_version: "encryptionkey@0"
I20260924 23:13:22.692219 26828 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:22.693380 26843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:22.693673 26828 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:22.693781 26828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "99a72e374aa044f0afbc5774ba21a5ec"
format_stamp: "Formatted at 2026-09-24 23:13:22 on dist-test-slave-m0z6"
server_key: "ecfc3bea0cc73bcc15b9dfb5b5813a80"
server_key_iv: "c13fc02a10790ef7a8f932b0ebf618ef"
server_key_version: "encryptionkey@0"
I20260924 23:13:22.693866 26828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:22.706527 26828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:22.709275 26828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:22.709444 26828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:22.713573 26828 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:42995
I20260924 23:13:22.713617 26895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:42995 every 8 connection(s)
I20260924 23:13:22.713990 26828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:13:22.715068 26896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:22.716832 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 26828
I20260924 23:13:22.716930 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:13:22.717115 13136 external_mini_cluster.cc:1428] Setting key c6d611c026ed11e63f93f59f9fab10aa
I20260924 23:13:22.717629 26896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec: Bootstrap starting.
I20260924 23:13:22.718398 26896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec: Neither blocks nor log segments found. Creating new log.
I20260924 23:13:22.718806 26896 log.cc:824] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:22.719630 26896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec: No bootstrap required, opened a new log
I20260924 23:13:22.722191 26896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a72e374aa044f0afbc5774ba21a5ec" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42995 } }
I20260924 23:13:22.722416 26896 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:13:22.722448 26896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a72e374aa044f0afbc5774ba21a5ec, State: Initialized, Role: FOLLOWER
I20260924 23:13:22.722587 26896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a72e374aa044f0afbc5774ba21a5ec" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42995 } }
I20260924 23:13:22.722721 26896 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:13:22.722760 26896 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:13:22.722805 26896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:13:22.723639 26896 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a72e374aa044f0afbc5774ba21a5ec" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42995 } }
I20260924 23:13:22.723788 26896 leader_election.cc:302] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99a72e374aa044f0afbc5774ba21a5ec; no voters:
I20260924 23:13:22.723996 26896 leader_election.cc:288] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:13:22.724081 26901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:13:22.724311 26901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 1 LEADER]: Becoming Leader. State: Replica: 99a72e374aa044f0afbc5774ba21a5ec, State: Running, Role: LEADER
I20260924 23:13:22.724459 26901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a72e374aa044f0afbc5774ba21a5ec" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42995 } }
I20260924 23:13:22.724327 26896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:13:22.724879 26902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99a72e374aa044f0afbc5774ba21a5ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a72e374aa044f0afbc5774ba21a5ec" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42995 } } }
I20260924 23:13:22.724974 26902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [sys.catalog]: This master's current role is: LEADER
Sep 24 23:13:22 dist-test-slave-m0z6 krb5kdc[26815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
W20260924 23:13:22.726717 26917 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:13:22.726774 26917 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:13:22.726112 26903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99a72e374aa044f0afbc5774ba21a5ec. Latest consensus state: current_term: 1 leader_uuid: "99a72e374aa044f0afbc5774ba21a5ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a72e374aa044f0afbc5774ba21a5ec" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42995 } } }
I20260924 23:13:22.726830 26905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:13:22.726845 26903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [sys.catalog]: This master's current role is: LEADER
I20260924 23:13:22.726984 26905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:13:22.729898 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:22.717675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52420 (local address 127.12.212.62:42995)
0924 23:13:22.717859 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:22.717870 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:22.717901 (+ 31us) server_negotiation.cc:407] Connection header received
0924 23:13:22.718569 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:22.718584 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:22.718676 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:22.718889 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:22.719685 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:22.721254 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:22.722148 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:22.722825 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:22.722946 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:22.725833 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:22.725838 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:22.725839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:22.725854 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:22.727985 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:22.728452 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:22.728455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:22.728455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:22.728608 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:22.728841 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:22.728841 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:22.728842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:22.729097 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:22.729215 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:22.729480 (+ 265us) server_negotiation.cc:299] Negotiation successful
0924 23:13:22.729642 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260924 23:13:22.731112 26905 catalog_manager.cc:1347] Generated new cluster ID: c182d3ed9e2346ec920bba11a8df7737
I20260924 23:13:22.731173 26905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:13:22.746304 26905 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:13:22.746965 26905 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:13:22.749907 26905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec: Generated new TSK 0
I20260924 23:13:22.750108 26905 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
I20260924 23:13:22.781183 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42995
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:13:22.863986 26922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:22.864225 26922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:22.864252 26922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:22.865501 26922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:22.865554 26922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:22.865583 26922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:13:22.867233 26922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:22.867483 26922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:22.867738 26922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:22.873003 26930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:22.873024 26928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:22.873019 26927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:22.873553 26922 server_base.cc:1034] running on GCE node
I20260924 23:13:22.873772 26922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:22.874002 26922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:22.875190 26922 hybrid_clock.cc:648] HybridClock initialized: now 1790291602875176 us; error 33 us; skew 500 ppm
Sep 24 23:13:22 dist-test-slave-m0z6 krb5kdc[26815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291602, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:22.877911 26922 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:13:22.878396 26922 webserver.cc:460] Webserver started at http://127.12.212.1:44821/ using document root <none> and password file <none>
I20260924 23:13:22.878609 26922 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:22.878659 26922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:22.878832 26922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:13:22.879959 26922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "add5c68ef5144558b24747730bf38f0a"
format_stamp: "Formatted at 2026-09-24 23:13:22 on dist-test-slave-m0z6"
server_key: "2a1ae48a1c164fa6c42908a34570ad88"
server_key_iv: "6524ac436028fd6a19396c10e27244a0"
server_key_version: "encryptionkey@0"
I20260924 23:13:22.880302 26922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "add5c68ef5144558b24747730bf38f0a"
format_stamp: "Formatted at 2026-09-24 23:13:22 on dist-test-slave-m0z6"
server_key: "2a1ae48a1c164fa6c42908a34570ad88"
server_key_iv: "6524ac436028fd6a19396c10e27244a0"
server_key_version: "encryptionkey@0"
I20260924 23:13:22.881650 26922 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:13:22.882397 26937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:22.882568 26922 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:13:22.882653 26922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "add5c68ef5144558b24747730bf38f0a"
format_stamp: "Formatted at 2026-09-24 23:13:22 on dist-test-slave-m0z6"
server_key: "2a1ae48a1c164fa6c42908a34570ad88"
server_key_iv: "6524ac436028fd6a19396c10e27244a0"
server_key_version: "encryptionkey@0"
I20260924 23:13:22.882756 26922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:22.894385 26922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:22.896791 26922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:22.896950 26922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:22.897185 26922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:13:22.897526 26922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:13:22.897562 26922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:22.897594 26922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:13:22.897619 26922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:22.903400 26922 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:42655
I20260924 23:13:22.903458 27050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:42655 every 8 connection(s)
I20260924 23:13:22.903764 26922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:13:22.906095 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 26922
I20260924 23:13:22.906185 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:13:22.906347 13136 external_mini_cluster.cc:1428] Setting key 0030cea0363c658cee0322896f5a87a2
Sep 24 23:13:22 dist-test-slave-m0z6 krb5kdc[26815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291602, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:22.914407 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:22.904448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:49857 (local address 127.12.212.62:42995)
0924 23:13:22.904503 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:22.904505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:22.904741 (+ 236us) server_negotiation.cc:407] Connection header received
0924 23:13:22.905245 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:22.905246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:22.905260 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:22.905312 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:22.906285 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:22.907020 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:22.908494 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:22.908799 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:22.908873 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:22.911260 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:22.911262 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:22.911263 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:22.911272 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:22.912810 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:22.913270 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:22.913272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:22.913273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:22.913339 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:22.913646 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:22.913647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:22.913648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:22.913803 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:22.913877 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:22.914213 (+ 336us) server_negotiation.cc:299] Negotiation successful
0924 23:13:22.914305 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 23:13:22.914505 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:22.904441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:49857)
0924 23:13:22.904609 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:22.904648 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:22.905143 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:22.905384 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:22.905388 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:22.905966 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:22.906183 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:22.906189 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:22.907139 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:22.907140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:22.908391 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:22.908395 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:22.908887 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:22.908888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:22.909001 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:22.909525 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:22.909535 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:22.911159 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:22.912927 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:22.912938 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:22.912941 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:22.913192 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:22.913440 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:22.913441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:22.913442 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:22.913574 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:22.913905 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:22.913913 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:22.914068 (+ 155us) client_negotiation.cc:769] Sending connection context
0924 23:13:22.914165 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 23:13:22.914307 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":61,"threads_started":1}
WARNING: no policy specified for kudu/127.12.212.2@KRBTEST.COM; defaulting to no policy
I20260924 23:13:22.915086 27051 heartbeater.cc:347] Connected to a master server at 127.12.212.62:42995
I20260924 23:13:22.915195 27051 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:22.915382 27051 heartbeater.cc:511] Master 127.12.212.62:42995 requested a full tablet report, sending...
I20260924 23:13:22.915992 26860 ts_manager.cc:194] Registered new tserver with Master: add5c68ef5144558b24747730bf38f0a (127.12.212.1:42655)
I20260924 23:13:22.916605 26860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:49857
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
I20260924 23:13:22.934158 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42995
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:13:23.014240 27056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:23.014488 27056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:23.014523 27056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:23.015838 27056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:23.015892 27056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:23.015930 27056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:13:23.017506 27056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:23.017793 27056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:23.018026 27056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:13:23.023273 27056 server_base.cc:1034] running on GCE node
W20260924 23:13:23.023283 27062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:23.023283 27064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:23.023273 27061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:23.023801 27056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:23.024058 27056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:23.025240 27056 hybrid_clock.cc:648] HybridClock initialized: now 1790291603025226 us; error 31 us; skew 500 ppm
Sep 24 23:13:23 dist-test-slave-m0z6 krb5kdc[26815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291603, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:23.028219 27056 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:13:23.028795 27056 webserver.cc:460] Webserver started at http://127.12.212.2:37661/ using document root <none> and password file <none>
I20260924 23:13:23.029035 27056 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:23.029088 27056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:23.029223 27056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:13:23.030337 27056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "4f92be39417149649ce5b36808fbd57f"
format_stamp: "Formatted at 2026-09-24 23:13:23 on dist-test-slave-m0z6"
server_key: "aa402f4d7e9a877788c8acc59299e50f"
server_key_iv: "b6e53b246584c718f16b9ec3149d0291"
server_key_version: "encryptionkey@0"
I20260924 23:13:23.030747 27056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "4f92be39417149649ce5b36808fbd57f"
format_stamp: "Formatted at 2026-09-24 23:13:23 on dist-test-slave-m0z6"
server_key: "aa402f4d7e9a877788c8acc59299e50f"
server_key_iv: "b6e53b246584c718f16b9ec3149d0291"
server_key_version: "encryptionkey@0"
I20260924 23:13:23.032392 27056 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:13:23.033334 27071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:23.033555 27056 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:13:23.033654 27056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "4f92be39417149649ce5b36808fbd57f"
format_stamp: "Formatted at 2026-09-24 23:13:23 on dist-test-slave-m0z6"
server_key: "aa402f4d7e9a877788c8acc59299e50f"
server_key_iv: "b6e53b246584c718f16b9ec3149d0291"
server_key_version: "encryptionkey@0"
I20260924 23:13:23.033736 27056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:23.044829 27056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:23.046941 27056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:23.047086 27056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:23.047300 27056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:13:23.047601 27056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:13:23.047634 27056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:23.047662 27056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:13:23.047679 27056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:23.053221 27056 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:40723
I20260924 23:13:23.053308 27184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:40723 every 8 connection(s)
I20260924 23:13:23.053603 27056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:13:23.058907 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 27056
I20260924 23:13:23.059029 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:13:23.059216 13136 external_mini_cluster.cc:1428] Setting key 806a056754b0ad5da2e286efb8b3cf25
Sep 24 23:13:23 dist-test-slave-m0z6 krb5kdc[26815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291603, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:23.063316 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.054391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:43109 (local address 127.12.212.62:42995)
0924 23:13:23.054563 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:23.054565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:23.054718 (+ 153us) server_negotiation.cc:407] Connection header received
0924 23:13:23.055284 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:23.055286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:23.055299 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:23.055351 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:23.056351 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.056862 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.057877 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.058087 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.058145 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.060574 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:23.060575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:23.060576 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:23.060584 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:23.061920 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:23.062307 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:23.062308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:23.062309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:23.062363 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:23.062590 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:23.062591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:23.062591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:23.062741 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:23.062805 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:23.063136 (+ 331us) server_negotiation.cc:299] Negotiation successful
0924 23:13:23.063222 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 23:13:23.063498 27187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.054311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:43109)
0924 23:13:23.054575 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:23.054627 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:23.055178 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:23.055429 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:23.055432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:23.056030 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:23.056240 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.056246 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.056955 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.056957 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.057784 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.057786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.058164 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.058165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.058295 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.058844 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:23.058855 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:23.060489 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:23.062005 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:23.062012 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:23.062015 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:23.062238 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:23.062420 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:23.062420 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:23.062421 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:23.062532 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:23.062802 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:23.062810 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:23.062976 (+ 166us) client_negotiation.cc:769] Sending connection context
0924 23:13:23.063093 (+ 117us) client_negotiation.cc:240] Negotiation successful
0924 23:13:23.063269 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":90,"threads_started":1}
I20260924 23:13:23.064246 27185 heartbeater.cc:347] Connected to a master server at 127.12.212.62:42995
I20260924 23:13:23.064340 27185 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:23.064569 27185 heartbeater.cc:511] Master 127.12.212.62:42995 requested a full tablet report, sending...
I20260924 23:13:23.064986 26860 ts_manager.cc:194] Registered new tserver with Master: 4f92be39417149649ce5b36808fbd57f (127.12.212.2:40723)
I20260924 23:13:23.065359 26860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:43109
WARNING: no policy specified for kudu/127.12.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
I20260924 23:13:23.086321 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42995
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:13:23.168262 27190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:23.168506 27190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:23.168540 27190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:23.169754 27190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:23.169801 27190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:23.169836 27190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:13:23.171366 27190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:23.171627 27190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:23.171854 27190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:23.177091 27195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:23.177099 27198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:23.177095 27196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:23.177188 27190 server_base.cc:1034] running on GCE node
I20260924 23:13:23.177572 27190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:23.177896 27190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:23.179073 27190 hybrid_clock.cc:648] HybridClock initialized: now 1790291603179050 us; error 35 us; skew 500 ppm
Sep 24 23:13:23 dist-test-slave-m0z6 krb5kdc[26815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291603, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:23.181842 27190 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:13:23.182389 27190 webserver.cc:460] Webserver started at http://127.12.212.3:45431/ using document root <none> and password file <none>
I20260924 23:13:23.182672 27190 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:23.182771 27190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:23.182916 27190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:13:23.184027 27190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "394b5678af3a476dbc8b8455e39c8be8"
format_stamp: "Formatted at 2026-09-24 23:13:23 on dist-test-slave-m0z6"
server_key: "13e27d3c3198054c4a4fa2912809bf6e"
server_key_iv: "aa6bba64170ebc082e31182c7cf3af81"
server_key_version: "encryptionkey@0"
I20260924 23:13:23.184382 27190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "394b5678af3a476dbc8b8455e39c8be8"
format_stamp: "Formatted at 2026-09-24 23:13:23 on dist-test-slave-m0z6"
server_key: "13e27d3c3198054c4a4fa2912809bf6e"
server_key_iv: "aa6bba64170ebc082e31182c7cf3af81"
server_key_version: "encryptionkey@0"
I20260924 23:13:23.185863 27190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:13:23.186741 27205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:23.186913 27190 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:23.186998 27190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "394b5678af3a476dbc8b8455e39c8be8"
format_stamp: "Formatted at 2026-09-24 23:13:23 on dist-test-slave-m0z6"
server_key: "13e27d3c3198054c4a4fa2912809bf6e"
server_key_iv: "aa6bba64170ebc082e31182c7cf3af81"
server_key_version: "encryptionkey@0"
I20260924 23:13:23.187067 27190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:23.219818 27190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:23.222368 27190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:23.222491 27190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:23.222716 27190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:13:23.223069 27190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:13:23.223109 27190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:23.223133 27190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:13:23.223155 27190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:23.229076 27190 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:42059
I20260924 23:13:23.229151 27318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:42059 every 8 connection(s)
I20260924 23:13:23.229445 27190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:13:23.231794 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 27190
I20260924 23:13:23.231876 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:13:23.232085 13136 external_mini_cluster.cc:1428] Setting key 39c857161bb22f66606588bb02239544
Sep 24 23:13:23 dist-test-slave-m0z6 krb5kdc[26815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291603, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:23.239182 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.230274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:36083 (local address 127.12.212.62:42995)
0924 23:13:23.230339 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:23.230342 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:23.230550 (+ 208us) server_negotiation.cc:407] Connection header received
0924 23:13:23.231190 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:23.231191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:23.231202 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:23.231288 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:23.232401 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.232908 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.234000 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.234202 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.234258 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.236534 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:23.236536 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:23.236536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:23.236545 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:23.237789 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:23.238215 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:23.238216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:23.238216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:23.238275 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:23.238495 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:23.238496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:23.238496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:23.238656 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:23.238735 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:23.239000 (+ 265us) server_negotiation.cc:299] Negotiation successful
0924 23:13:23.239086 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 23:13:23.239379 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.230245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:36083)
0924 23:13:23.230415 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:23.230460 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:23.231058 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:23.231366 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:23.231371 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:23.232036 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:23.232305 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.232313 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.232995 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.232996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.233918 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.233921 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.234273 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.234274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.234401 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.234982 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:23.234991 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:23.236436 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:23.237873 (+ 1437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:23.237882 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:23.237884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:23.238141 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:23.238330 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:23.238330 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:23.238331 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:23.238439 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:23.238726 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:23.238733 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:23.238880 (+ 147us) client_negotiation.cc:769] Sending connection context
0924 23:13:23.238973 (+ 93us) client_negotiation.cc:240] Negotiation successful
0924 23:13:23.239155 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260924 23:13:23.239902 27319 heartbeater.cc:347] Connected to a master server at 127.12.212.62:42995
I20260924 23:13:23.239993 27319 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:23.240192 27319 heartbeater.cc:511] Master 127.12.212.62:42995 requested a full tablet report, sending...
I20260924 23:13:23.240554 26860 ts_manager.cc:194] Registered new tserver with Master: 394b5678af3a476dbc8b8455e39c8be8 (127.12.212.3:42059)
I20260924 23:13:23.240911 26860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:36083
I20260924 23:13:23.243790 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 23:13:23.251657 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.245151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52430 (local address 127.12.212.62:42995)
0924 23:13:23.245209 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:23.245210 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:23.245303 (+ 93us) server_negotiation.cc:407] Connection header received
0924 23:13:23.245348 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:23.245349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:23.245359 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:23.245408 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:23.245905 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.246393 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.247230 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.247429 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.247546 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.248804 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:23.248806 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:23.248807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:23.248814 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:23.250156 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:23.250589 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:23.250590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:23.250590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:23.250832 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:23.251089 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:23.251090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:23.251091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:23.251268 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:23.251352 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:23.251505 (+ 153us) server_negotiation.cc:299] Negotiation successful
0924 23:13:23.251584 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 23:13:23.253263 26860 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:13:23.254854 26860 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:13:23.261423 27333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.257406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:42059 (local address 127.0.0.1:55098)
0924 23:13:23.257774 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:23.257783 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:23.257832 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:23.258052 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:23.258053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:23.258055 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:13:23.258110 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.258112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.259356 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.259357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.260280 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.260283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.261117 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.261118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.261193 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.261216 (+ 23us) client_negotiation.cc:769] Sending connection context
0924 23:13:23.261283 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 23:13:23.261338 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":73,"threads_started":1}
I20260924 23:13:23.261439 27335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.257609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55098 (local address 127.12.212.3:42059)
0924 23:13:23.257777 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:23.257783 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:23.257823 (+ 40us) server_negotiation.cc:407] Connection header received
0924 23:13:23.257890 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:23.257892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:23.257906 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:23.257984 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:13:23.258174 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.259255 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.260364 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.261036 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.261110 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.261170 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:23.261308 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 23:13:23.261348 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260924 23:13:23.261591 27334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.257368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42024 (local address 127.12.212.2:40723)
0924 23:13:23.257954 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:23.257959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:23.257967 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:13:23.258006 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:23.258008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:23.258021 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:23.258332 (+ 311us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:13:23.258407 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.259131 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.260654 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.261149 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.261210 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.261265 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:23.261473 (+ 208us) server_negotiation.cc:299] Negotiation successful
0924 23:13:23.261514 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":163,"threads_started":1}
I20260924 23:13:23.261803 27330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.257206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:42655 (local address 127.0.0.1:42532)
0924 23:13:23.257516 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:23.257563 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:23.257638 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:23.257829 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:23.257832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:23.257834 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:13:23.257914 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.257916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.259320 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.259321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.260525 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.260527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.261415 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.261416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.261477 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.261481 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:13:23.261694 (+ 213us) client_negotiation.cc:240] Negotiation successful
0924 23:13:23.261736 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":34,"thread_start_us":60,"threads_started":1}
I20260924 23:13:23.261986 27331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.257241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:40723 (local address 127.0.0.1:42024)
0924 23:13:23.257637 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:23.257648 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:23.257706 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:23.258227 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:23.258227 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:23.258229 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:13:23.258262 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.258264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.259216 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.259218 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.260599 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.260601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.261295 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.261296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.261348 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.261351 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:13:23.261391 (+ 40us) client_negotiation.cc:240] Negotiation successful
0924 23:13:23.261423 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":64,"thread_start_us":81,"threads_started":1}
I20260924 23:13:23.262149 27332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.257305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42532 (local address 127.12.212.1:42655)
0924 23:13:23.257527 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:23.257533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:23.257626 (+ 93us) server_negotiation.cc:407] Connection header received
0924 23:13:23.257731 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:23.257734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:23.257751 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:23.257825 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:13:23.257990 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.259162 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.260619 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.261323 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.261929 (+ 606us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.262000 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:23.262024 (+ 24us) server_negotiation.cc:299] Negotiation successful
0924 23:13:23.262064 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":85,"threads_started":1}
I20260924 23:13:23.262235 27253 tablet_service.cc:1467] Processing CreateTablet for tablet fb27b1fbf083431d919d5abe5ea17a7c (DEFAULT_TABLE table=test-table [id=6234378b9bc443079ed8cd88fb90b9c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:13:23.262611 27253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb27b1fbf083431d919d5abe5ea17a7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:23.262601 27119 tablet_service.cc:1467] Processing CreateTablet for tablet fb27b1fbf083431d919d5abe5ea17a7c (DEFAULT_TABLE table=test-table [id=6234378b9bc443079ed8cd88fb90b9c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:13:23.262784 26985 tablet_service.cc:1467] Processing CreateTablet for tablet fb27b1fbf083431d919d5abe5ea17a7c (DEFAULT_TABLE table=test-table [id=6234378b9bc443079ed8cd88fb90b9c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:13:23.263031 27119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb27b1fbf083431d919d5abe5ea17a7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:23.263118 26985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb27b1fbf083431d919d5abe5ea17a7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:23.265561 27336 tablet_bootstrap.cc:492] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a: Bootstrap starting.
I20260924 23:13:23.266158 27338 tablet_bootstrap.cc:492] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f: Bootstrap starting.
I20260924 23:13:23.266242 27336 tablet_bootstrap.cc:654] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a: Neither blocks nor log segments found. Creating new log.
I20260924 23:13:23.266397 27337 tablet_bootstrap.cc:492] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8: Bootstrap starting.
I20260924 23:13:23.266831 27336 log.cc:824] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:23.266973 27337 tablet_bootstrap.cc:654] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8: Neither blocks nor log segments found. Creating new log.
I20260924 23:13:23.267082 27338 tablet_bootstrap.cc:654] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f: Neither blocks nor log segments found. Creating new log.
I20260924 23:13:23.267417 27337 log.cc:824] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:23.267563 27338 log.cc:824] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:23.268165 27337 tablet_bootstrap.cc:492] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8: No bootstrap required, opened a new log
I20260924 23:13:23.268165 27336 tablet_bootstrap.cc:492] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a: No bootstrap required, opened a new log
I20260924 23:13:23.268180 27338 tablet_bootstrap.cc:492] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f: No bootstrap required, opened a new log
I20260924 23:13:23.268236 27338 ts_tablet_manager.cc:1397] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:13:23.268236 27337 ts_tablet_manager.cc:1397] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 23:13:23.268236 27336 ts_tablet_manager.cc:1397] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 23:13:23.269793 27338 raft_consensus.cc:348] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } }
I20260924 23:13:23.269932 27338 raft_consensus.cc:374] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:13:23.269961 27338 raft_consensus.cc:729] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f92be39417149649ce5b36808fbd57f, State: Initialized, Role: FOLLOWER
I20260924 23:13:23.270077 27338 consensus_queue.cc:260] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } }
I20260924 23:13:23.270136 27337 raft_consensus.cc:348] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } }
I20260924 23:13:23.270305 27337 raft_consensus.cc:374] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:13:23.270352 27337 raft_consensus.cc:729] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 394b5678af3a476dbc8b8455e39c8be8, State: Initialized, Role: FOLLOWER
I20260924 23:13:23.270345 27338 ts_tablet_manager.cc:1428] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:13:23.270310 27336 raft_consensus.cc:348] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } }
I20260924 23:13:23.270459 27336 raft_consensus.cc:374] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:13:23.270498 27336 raft_consensus.cc:729] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: add5c68ef5144558b24747730bf38f0a, State: Initialized, Role: FOLLOWER
I20260924 23:13:23.270466 27337 consensus_queue.cc:260] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } }
I20260924 23:13:23.270506 27185 heartbeater.cc:503] Master 127.12.212.62:42995 was elected leader, sending a full tablet report...
I20260924 23:13:23.270597 27336 consensus_queue.cc:260] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } }
I20260924 23:13:23.270758 27337 ts_tablet_manager.cc:1428] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260924 23:13:23.270877 27336 ts_tablet_manager.cc:1428] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:13:23.270890 27319 heartbeater.cc:503] Master 127.12.212.62:42995 was elected leader, sending a full tablet report...
I20260924 23:13:23.270923 27051 heartbeater.cc:503] Master 127.12.212.62:42995 was elected leader, sending a full tablet report...
W20260924 23:13:23.304203 27186 tablet.cc:2369] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:13:23.357034 27342 raft_consensus.cc:482] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:13:23.357164 27342 raft_consensus.cc:504] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } }
I20260924 23:13:23.357609 27342 leader_election.cc:288] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers add5c68ef5144558b24747730bf38f0a (127.12.212.1:42655), 394b5678af3a476dbc8b8455e39c8be8 (127.12.212.3:42059)
I20260924 23:13:23.360690 27335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.358055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:35579 (local address 127.12.212.3:42059)
0924 23:13:23.358100 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:23.358102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:23.358173 (+ 71us) server_negotiation.cc:407] Connection header received
0924 23:13:23.358223 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:23.358224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:23.358234 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:23.358286 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:13:23.358444 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.358967 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.359947 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.360338 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.360401 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.360412 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:23.360573 (+ 161us) server_negotiation.cc:299] Negotiation successful
0924 23:13:23.360617 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 23:13:23.360690 27332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.357906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:37895 (local address 127.12.212.1:42655)
0924 23:13:23.357961 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:23.357963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:23.357974 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:13:23.358088 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:23.358090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:23.358101 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:23.358162 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:13:23.358329 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.358890 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.359948 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.360345 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.360415 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.360424 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:23.360586 (+ 162us) server_negotiation.cc:299] Negotiation successful
0924 23:13:23.360625 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 23:13:23.360704 27187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.357826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:42655 (local address 127.12.212.2:37895)
0924 23:13:23.357888 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:23.357899 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:23.357967 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:23.358173 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:23.358175 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:23.358177 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:13:23.358252 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.358254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.358966 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.358967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.359840 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.359843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.360428 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.360429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.360506 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.360509 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:13:23.360569 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 23:13:23.360620 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":43}
I20260924 23:13:23.360705 27345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.357945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:42059 (local address 127.12.212.2:35579)
0924 23:13:23.358105 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:23.358113 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:23.358162 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:23.358304 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:23.358305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:23.358307 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:13:23.358379 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.358381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.359047 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.359048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.359840 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.359843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.360421 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.360422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.360498 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.360502 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:13:23.360558 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 23:13:23.360604 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":2,"thread_start_us":139,"threads_started":1}
I20260924 23:13:23.361186 27005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb27b1fbf083431d919d5abe5ea17a7c" candidate_uuid: "4f92be39417149649ce5b36808fbd57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "add5c68ef5144558b24747730bf38f0a" is_pre_election: true
I20260924 23:13:23.361184 27273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb27b1fbf083431d919d5abe5ea17a7c" candidate_uuid: "4f92be39417149649ce5b36808fbd57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "394b5678af3a476dbc8b8455e39c8be8" is_pre_election: true
I20260924 23:13:23.361363 27273 raft_consensus.cc:2454] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f92be39417149649ce5b36808fbd57f in term 0.
I20260924 23:13:23.361363 27005 raft_consensus.cc:2454] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f92be39417149649ce5b36808fbd57f in term 0.
I20260924 23:13:23.361616 27074 leader_election.cc:302] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 394b5678af3a476dbc8b8455e39c8be8, 4f92be39417149649ce5b36808fbd57f; no voters:
I20260924 23:13:23.361780 27342 raft_consensus.cc:2781] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:13:23.361907 27342 raft_consensus.cc:482] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:13:23.361945 27342 raft_consensus.cc:3037] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:13:23.362666 27342 raft_consensus.cc:504] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } }
I20260924 23:13:23.362830 27342 leader_election.cc:288] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [CANDIDATE]: Term 1 election: Requested vote from peers add5c68ef5144558b24747730bf38f0a (127.12.212.1:42655), 394b5678af3a476dbc8b8455e39c8be8 (127.12.212.3:42059)
I20260924 23:13:23.362989 27005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb27b1fbf083431d919d5abe5ea17a7c" candidate_uuid: "4f92be39417149649ce5b36808fbd57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "add5c68ef5144558b24747730bf38f0a"
I20260924 23:13:23.363039 27273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb27b1fbf083431d919d5abe5ea17a7c" candidate_uuid: "4f92be39417149649ce5b36808fbd57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "394b5678af3a476dbc8b8455e39c8be8"
I20260924 23:13:23.363077 27005 raft_consensus.cc:3037] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:13:23.363143 27273 raft_consensus.cc:3037] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:13:23.363971 27273 raft_consensus.cc:2454] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f92be39417149649ce5b36808fbd57f in term 1.
I20260924 23:13:23.363971 27005 raft_consensus.cc:2454] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f92be39417149649ce5b36808fbd57f in term 1.
I20260924 23:13:23.364172 27074 leader_election.cc:302] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f92be39417149649ce5b36808fbd57f, add5c68ef5144558b24747730bf38f0a; no voters:
I20260924 23:13:23.364259 27342 raft_consensus.cc:2781] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:13:23.364406 27342 raft_consensus.cc:686] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [term 1 LEADER]: Becoming Leader. State: Replica: 4f92be39417149649ce5b36808fbd57f, State: Running, Role: LEADER
I20260924 23:13:23.364492 27342 consensus_queue.cc:237] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } }
I20260924 23:13:23.365357 26859 catalog_manager.cc:5495] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f92be39417149649ce5b36808fbd57f (127.12.212.2). New cstate: current_term: 1 leader_uuid: "4f92be39417149649ce5b36808fbd57f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } health_report { overall_health: HEALTHY } } }
I20260924 23:13:23.381219 27334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.378781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42038 (local address 127.12.212.2:40723)
0924 23:13:23.378832 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:23.378834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:23.378844 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:13:23.378886 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:23.378887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:23.378906 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:23.378983 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:13:23.379131 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.379624 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.380465 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.380708 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.380771 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.380932 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:13:23.381040 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:13:23.381105 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:23.381117 (+ 12us) server_negotiation.cc:299] Negotiation successful
0924 23:13:23.381136 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 23:13:23.383260 27273 raft_consensus.cc:1261] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8 [term 1 FOLLOWER]: Refusing update from remote peer 4f92be39417149649ce5b36808fbd57f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:13:23.383280 27005 raft_consensus.cc:1261] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a [term 1 FOLLOWER]: Refusing update from remote peer 4f92be39417149649ce5b36808fbd57f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:13:23.383580 27342 consensus_queue.cc:1035] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [LEADER]: Connected to new peer: Peer: permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:13:23.383720 27346 consensus_queue.cc:1035] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [LEADER]: Connected to new peer: Peer: permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:13:23.386126 27350 mvcc.cc:205] Tried to move back new op lower bound from 7333034407454765056 to 7333034407381504000. Current Snapshot: MvccSnapshot[applied={T|T < 7333034407454765056}]
I20260924 23:13:23.388792 26860 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:13:23.388890 26860 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:13:23.389712 26860 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec: Sending DeleteTablet for 3 replicas of tablet fb27b1fbf083431d919d5abe5ea17a7c
I20260924 23:13:23.390123 27253 tablet_service.cc:1514] Processing DeleteTablet for tablet fb27b1fbf083431d919d5abe5ea17a7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:13:23 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:55098
I20260924 23:13:23.390125 26985 tablet_service.cc:1514] Processing DeleteTablet for tablet fb27b1fbf083431d919d5abe5ea17a7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:13:23 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:42532
I20260924 23:13:23.390156 27119 tablet_service.cc:1514] Processing DeleteTablet for tablet fb27b1fbf083431d919d5abe5ea17a7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:13:23 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:42024
I20260924 23:13:23.390362 27361 tablet_replica.cc:330] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f: stopping tablet replica
I20260924 23:13:23.390455 27361 raft_consensus.cc:2229] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:13:23.390522 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 26922
I20260924 23:13:23.390583 27361 raft_consensus.cc:2258] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:13:23.391683 27359 tablet_replica.cc:330] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8: stopping tablet replica
I20260924 23:13:23.391836 27361 ts_tablet_manager.cc:1905] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:13:23.391863 27359 raft_consensus.cc:2229] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:13:23.392055 27359 raft_consensus.cc:2258] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:13:23.392531 27359 ts_tablet_manager.cc:1905] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:13:23.393689 27361 ts_tablet_manager.cc:1918] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:13:23.393769 27361 log.cc:1196] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/fb27b1fbf083431d919d5abe5ea17a7c
I20260924 23:13:23.394057 27361 ts_tablet_manager.cc:1939] T fb27b1fbf083431d919d5abe5ea17a7c P 4f92be39417149649ce5b36808fbd57f: Deleting consensus metadata
I20260924 23:13:23.394320 27359 ts_tablet_manager.cc:1918] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:13:23.394445 27359 log.cc:1196] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/fb27b1fbf083431d919d5abe5ea17a7c
I20260924 23:13:23.394505 26847 catalog_manager.cc:4841] TS 4f92be39417149649ce5b36808fbd57f (127.12.212.2:40723): tablet fb27b1fbf083431d919d5abe5ea17a7c (table test-table [id=6234378b9bc443079ed8cd88fb90b9c3]) successfully deleted
I20260924 23:13:23.394829 27359 ts_tablet_manager.cc:1939] T fb27b1fbf083431d919d5abe5ea17a7c P 394b5678af3a476dbc8b8455e39c8be8: Deleting consensus metadata
I20260924 23:13:23.395256 26846 catalog_manager.cc:4841] TS 394b5678af3a476dbc8b8455e39c8be8 (127.12.212.3:42059): tablet fb27b1fbf083431d919d5abe5ea17a7c (table test-table [id=6234378b9bc443079ed8cd88fb90b9c3]) successfully deleted
W20260924 23:13:23.396222 26846 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.212.1:42655 (error 108)
I20260924 23:13:23.396765 27331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.396632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:42655 (local address 127.0.0.1:42534)
0924 23:13:23.396689 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:23.396731 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.1:42655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260924 23:13:23.396765 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 27056
W20260924 23:13:23.397470 26846 catalog_manager.cc:4584] TS add5c68ef5144558b24747730bf38f0a (127.12.212.1:42655): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fb27b1fbf083431d919d5abe5ea17a7c: Network error: Client connection negotiation failed: client connection to 127.12.212.1:42655: connect: Connection refused (error 111)
I20260924 23:13:23.400940 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 27190
I20260924 23:13:23.405347 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 26828
I20260924 23:13:23.409883 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:42995
--webserver_interface=127.12.212.62
--webserver_port=35103
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:42995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:13:23.492308 27363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:23.492561 27363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:23.492585 27363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:23.493800 27363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:13:23.493842 27363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:23.493855 27363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:23.493866 27363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:13:23.493877 27363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:13:23.495388 27363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:42995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:42995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=35103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:23.495630 27363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:23.495844 27363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:23.503964 27372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:23.503964 27370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:23.504027 27369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:23.504262 27363 server_base.cc:1034] running on GCE node
I20260924 23:13:23.504527 27363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:23.504848 27363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:23.506003 27363 hybrid_clock.cc:648] HybridClock initialized: now 1790291603505978 us; error 37 us; skew 500 ppm
Sep 24 23:13:23 dist-test-slave-m0z6 krb5kdc[26815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291603, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:23.512295 27363 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:13:23.512880 27363 webserver.cc:460] Webserver started at http://127.12.212.62:35103/ using document root <none> and password file <none>
I20260924 23:13:23.513119 27363 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:23.513188 27363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:23.514731 27363 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 23:13:23.515421 27379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:23.515625 27363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:13:23.515717 27363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "99a72e374aa044f0afbc5774ba21a5ec"
format_stamp: "Formatted at 2026-09-24 23:13:22 on dist-test-slave-m0z6"
server_key: "ecfc3bea0cc73bcc15b9dfb5b5813a80"
server_key_iv: "c13fc02a10790ef7a8f932b0ebf618ef"
server_key_version: "encryptionkey@0"
I20260924 23:13:23.516141 27363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:23.543280 27363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:23.545564 27363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:23.545706 27363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:23.549530 27363 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:42995
I20260924 23:13:23.549564 27431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:42995 every 8 connection(s)
I20260924 23:13:23.549945 27363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:13:23.551614 27432 sys_catalog.cc:263] Verifying existing consensus state
I20260924 23:13:23.552237 27432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec: Bootstrap starting.
I20260924 23:13:23.555145 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 27363
I20260924 23:13:23.555397 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:42655
--local_ip_for_outbound_sockets=127.12.212.1
--tserver_master_addrs=127.12.212.62:42995
--webserver_port=44821
--webserver_interface=127.12.212.1
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 23:13:23.560421 27432 log.cc:824] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:23.562594 27432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 23:13:23.562932 27432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec: Bootstrap complete.
I20260924 23:13:23.565088 27432 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a72e374aa044f0afbc5774ba21a5ec" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42995 } }
I20260924 23:13:23.565327 27432 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a72e374aa044f0afbc5774ba21a5ec, State: Initialized, Role: FOLLOWER
I20260924 23:13:23.565477 27432 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a72e374aa044f0afbc5774ba21a5ec" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42995 } }
I20260924 23:13:23.565553 27432 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:13:23.565601 27432 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:13:23.565650 27432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 1 FOLLOWER]: Advancing to term 2
I20260924 23:13:23.566756 27432 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a72e374aa044f0afbc5774ba21a5ec" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42995 } }
I20260924 23:13:23.566859 27432 leader_election.cc:302] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99a72e374aa044f0afbc5774ba21a5ec; no voters:
I20260924 23:13:23.567034 27432 leader_election.cc:288] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 23:13:23.567085 27436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 2 FOLLOWER]: Leader election won for term 2
I20260924 23:13:23.567231 27436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 2 LEADER]: Becoming Leader. State: Replica: 99a72e374aa044f0afbc5774ba21a5ec, State: Running, Role: LEADER
I20260924 23:13:23.567314 27432 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:13:23.567498 27436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a72e374aa044f0afbc5774ba21a5ec" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42995 } }
I20260924 23:13:23.567839 27436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99a72e374aa044f0afbc5774ba21a5ec. Latest consensus state: current_term: 2 leader_uuid: "99a72e374aa044f0afbc5774ba21a5ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a72e374aa044f0afbc5774ba21a5ec" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42995 } } }
I20260924 23:13:23.567929 27436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [sys.catalog]: This master's current role is: LEADER
I20260924 23:13:23.568202 27437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "99a72e374aa044f0afbc5774ba21a5ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a72e374aa044f0afbc5774ba21a5ec" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42995 } } }
I20260924 23:13:23.568351 27437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [sys.catalog]: This master's current role is: LEADER
I20260924 23:13:23.568809 27445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:13:23.569741 27445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:13:23.569844 27445 catalog_manager.cc:1259] Loaded cluster ID: c182d3ed9e2346ec920bba11a8df7737
I20260924 23:13:23.569907 27445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:13:23.570161 27445 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260924 23:13:23.575171 27445 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260924 23:13:23.577752 27445 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:13:23.577973 27445 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec: Loaded TSK: 0
I20260924 23:13:23.578200 27445 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 23:13:23.640475 27434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:23.640710 27434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:23.640735 27434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:23.641966 27434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:23.642016 27434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:23.642043 27434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:13:23.643626 27434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:42655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=44821
--tserver_master_addrs=127.12.212.62:42995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:23.643880 27434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:23.644121 27434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:23.649025 27462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:23.649046 27459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:23.649014 27460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:23.649523 27434 server_base.cc:1034] running on GCE node
I20260924 23:13:23.649735 27434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:23.649968 27434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:23.651111 27434 hybrid_clock.cc:648] HybridClock initialized: now 1790291603651093 us; error 32 us; skew 500 ppm
Sep 24 23:13:23 dist-test-slave-m0z6 krb5kdc[26815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291603, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:23.653733 27434 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:13:23.654250 27434 webserver.cc:460] Webserver started at http://127.12.212.1:44821/ using document root <none> and password file <none>
I20260924 23:13:23.654498 27434 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:23.654619 27434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:23.656360 27434 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:23.657092 27469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:23.657250 27434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:13:23.657342 27434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "add5c68ef5144558b24747730bf38f0a"
format_stamp: "Formatted at 2026-09-24 23:13:22 on dist-test-slave-m0z6"
server_key: "2a1ae48a1c164fa6c42908a34570ad88"
server_key_iv: "6524ac436028fd6a19396c10e27244a0"
server_key_version: "encryptionkey@0"
I20260924 23:13:23.657783 27434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:23.688203 27434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:23.691066 27434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:23.691226 27434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:23.691488 27434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:13:23.692015 27476 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 23:13:23.693496 27434 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 23:13:23.693565 27434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 23:13:23.693604 27434 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 23:13:23.694530 27434 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 23:13:23.694576 27434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 23:13:23.694666 27476 tablet_bootstrap.cc:492] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a: Bootstrap starting.
I20260924 23:13:23.701166 27434 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:42655
I20260924 23:13:23.701372 27583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:42655 every 8 connection(s)
I20260924 23:13:23.701643 27434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:13:23.705304 27476 log.cc:824] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:23.707491 27476 tablet_bootstrap.cc:492] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 23:13:23.707748 27476 tablet_bootstrap.cc:492] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a: Bootstrap complete.
I20260924 23:13:23.708480 27476 ts_tablet_manager.cc:1397] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.008s
I20260924 23:13:23.709597 27476 raft_consensus.cc:348] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } }
I20260924 23:13:23.709830 27476 raft_consensus.cc:729] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: add5c68ef5144558b24747730bf38f0a, State: Initialized, Role: FOLLOWER
I20260924 23:13:23.709965 27476 consensus_queue.cc:260] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } }
I20260924 23:13:23.710242 27476 ts_tablet_manager.cc:1428] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
Sep 24 23:13:23 dist-test-slave-m0z6 krb5kdc[26815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291603, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:23.710668 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 27434
I20260924 23:13:23.710915 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:40723
--local_ip_for_outbound_sockets=127.12.212.2
--tserver_master_addrs=127.12.212.62:42995
--webserver_port=37661
--webserver_interface=127.12.212.2
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 23:13:23.714875 27587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.702524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:36503 (local address 127.12.212.62:42995)
0924 23:13:23.703861 (+ 1337us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:23.703871 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:23.703914 (+ 43us) server_negotiation.cc:407] Connection header received
0924 23:13:23.704063 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:23.704071 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:23.704224 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:23.704418 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:23.705532 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.706889 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.708045 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.708469 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.708565 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.711039 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:23.711044 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:23.711045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:23.711066 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:23.712923 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:23.713383 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:23.713387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:23.713387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:23.713491 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:23.713824 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:23.713825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:23.713826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:23.714110 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:23.714189 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:23.714494 (+ 305us) server_negotiation.cc:299] Negotiation successful
0924 23:13:23.714637 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1278,"thread_start_us":114,"threads_started":1}
I20260924 23:13:23.714875 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.702418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:36503)
0924 23:13:23.702824 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:23.702888 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:23.703609 (+ 721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:23.704486 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:23.704655 (+ 169us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:23.705037 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:23.705352 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.705359 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.707010 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.707011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.707954 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.707957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.708571 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.708572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.708721 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.709296 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:23.709304 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:23.710926 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:23.713044 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:23.713053 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:23.713055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:23.713301 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:23.713634 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:23.713635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:23.713635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:23.713756 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:23.714182 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:23.714189 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:23.714334 (+ 145us) client_negotiation.cc:769] Sending connection context
0924 23:13:23.714425 (+ 91us) client_negotiation.cc:240] Negotiation successful
0924 23:13:23.714636 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":103,"threads_started":1}
I20260924 23:13:23.715557 27584 heartbeater.cc:347] Connected to a master server at 127.12.212.62:42995
I20260924 23:13:23.715680 27584 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:23.715893 27584 heartbeater.cc:511] Master 127.12.212.62:42995 requested a full tablet report, sending...
I20260924 23:13:23.716692 27396 ts_manager.cc:194] Registered new tserver with Master: add5c68ef5144558b24747730bf38f0a (127.12.212.1:42655)
I20260924 23:13:23.717736 27396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:36503
I20260924 23:13:23.718601 27584 heartbeater.cc:503] Master 127.12.212.62:42995 was elected leader, sending a full tablet report...
Sep 24 23:13:23 dist-test-slave-m0z6 krb5kdc[26815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291603, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for kudu/127.12.212.1@KRBTEST.COM
I20260924 23:13:23.726368 27591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.717322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:42655 (local address 127.0.0.1:42536)
0924 23:13:23.717501 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:23.717541 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:23.717606 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:23.717978 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:23.717985 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:23.718333 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:23.718403 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.718406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.719466 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.719467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.720438 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.720443 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.720944 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.720945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.721042 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.721592 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:23.721598 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:23.723116 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:23.725189 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:23.725198 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:23.725200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:23.725413 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:23.725634 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:23.725636 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:23.725636 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:23.725705 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:23.726036 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:23.726043 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:23.726097 (+ 54us) client_negotiation.cc:769] Sending connection context
0924 23:13:23.726167 (+ 70us) client_negotiation.cc:240] Negotiation successful
0924 23:13:23.726278 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
I20260924 23:13:23.726410 27592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.717572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42536 (local address 127.12.212.1:42655)
0924 23:13:23.717745 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:23.717749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:23.717761 (+ 12us) server_negotiation.cc:407] Connection header received
0924 23:13:23.717798 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:23.717800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:23.717814 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:23.717889 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:23.718502 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.719391 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.720554 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.720850 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.720912 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.723207 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:23.723218 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:23.723218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:23.723227 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:23.725095 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:23.725483 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:23.725486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:23.725486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:23.725548 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:23.725763 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:23.725764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:23.725764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:23.725963 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:23.726038 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:23.726212 (+ 174us) server_negotiation.cc:299] Negotiation successful
0924 23:13:23.726310 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":73,"threads_started":1}
I20260924 23:13:23.726795 27518 tablet_service.cc:1514] Processing DeleteTablet for tablet fb27b1fbf083431d919d5abe5ea17a7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:13:23 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:42536
I20260924 23:13:23.727451 27593 tablet_replica.cc:330] stopping tablet replica
I20260924 23:13:23.727561 27593 raft_consensus.cc:2229] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:13:23.727630 27593 raft_consensus.cc:2258] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:13:23.727914 27593 ts_tablet_manager.cc:1905] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:13:23.729931 27593 ts_tablet_manager.cc:1918] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:13:23.730003 27593 log.cc:1196] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/wal/wals/fb27b1fbf083431d919d5abe5ea17a7c
I20260924 23:13:23.730180 27593 ts_tablet_manager.cc:1939] T fb27b1fbf083431d919d5abe5ea17a7c P add5c68ef5144558b24747730bf38f0a: Deleting consensus metadata
I20260924 23:13:23.730633 27382 catalog_manager.cc:4841] TS add5c68ef5144558b24747730bf38f0a (127.12.212.1:42655): tablet fb27b1fbf083431d919d5abe5ea17a7c (table test-table [id=6234378b9bc443079ed8cd88fb90b9c3]) successfully deleted
W20260924 23:13:23.796471 27590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:23.796697 27590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:23.796722 27590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:23.798018 27590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:23.798069 27590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:23.798096 27590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:13:23.799625 27590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:40723
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=37661
--tserver_master_addrs=127.12.212.62:42995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:23.799857 27590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:23.800107 27590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:23.805033 27598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:23.805081 27601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:23.805044 27599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:23.805362 27590 server_base.cc:1034] running on GCE node
I20260924 23:13:23.805585 27590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:23.805812 27590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:23.806947 27590 hybrid_clock.cc:648] HybridClock initialized: now 1790291603806926 us; error 34 us; skew 500 ppm
Sep 24 23:13:23 dist-test-slave-m0z6 krb5kdc[26815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291603, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:23.809433 27590 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:13:23.809846 27590 webserver.cc:460] Webserver started at http://127.12.212.2:37661/ using document root <none> and password file <none>
I20260924 23:13:23.810022 27590 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:23.810078 27590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:23.811429 27590 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260924 23:13:23.812057 27608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:23.812232 27590 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:23.812314 27590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "4f92be39417149649ce5b36808fbd57f"
format_stamp: "Formatted at 2026-09-24 23:13:23 on dist-test-slave-m0z6"
server_key: "aa402f4d7e9a877788c8acc59299e50f"
server_key_iv: "b6e53b246584c718f16b9ec3149d0291"
server_key_version: "encryptionkey@0"
I20260924 23:13:23.812685 27590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:23.826377 27590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:23.828480 27590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:23.828619 27590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:23.828855 27590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:13:23.829149 27590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:13:23.829185 27590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:23.829216 27590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:13:23.829233 27590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:23.834676 27590 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:40723
I20260924 23:13:23.834766 27721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:40723 every 8 connection(s)
I20260924 23:13:23.835068 27590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:13:23.836447 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 27590
I20260924 23:13:23.836725 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:42059
--local_ip_for_outbound_sockets=127.12.212.3
--tserver_master_addrs=127.12.212.62:42995
--webserver_port=45431
--webserver_interface=127.12.212.3
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 23:13:23 dist-test-slave-m0z6 krb5kdc[26815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291603, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:23.845093 27587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.835746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:54401 (local address 127.12.212.62:42995)
0924 23:13:23.835792 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:23.835794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:23.835987 (+ 193us) server_negotiation.cc:407] Connection header received
0924 23:13:23.836574 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:23.836575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:23.836592 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:23.836671 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:23.837827 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.838342 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.839401 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.839613 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.839672 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.842011 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:23.842013 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:23.842013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:23.842021 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:23.843680 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:23.844108 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:23.844109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:23.844110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:23.844167 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:23.844391 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:23.844392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:23.844393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:23.844509 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:23.844565 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:23.844872 (+ 307us) server_negotiation.cc:299] Negotiation successful
0924 23:13:23.844997 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 23:13:23.845266 27724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.835715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:54401)
0924 23:13:23.835867 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:23.835911 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:23.836408 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:23.836793 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:23.836798 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:23.837490 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:23.837706 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.837712 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.838439 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.838440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.839303 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.839306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.839686 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.839686 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.839808 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.840321 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:23.840330 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:23.841901 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:23.843748 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:23.843758 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:23.843761 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:23.844037 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:23.844220 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:23.844221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:23.844221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:23.844336 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:23.844564 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:23.844571 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:23.844737 (+ 166us) client_negotiation.cc:769] Sending connection context
0924 23:13:23.844845 (+ 108us) client_negotiation.cc:240] Negotiation successful
0924 23:13:23.845014 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
I20260924 23:13:23.845785 27722 heartbeater.cc:347] Connected to a master server at 127.12.212.62:42995
I20260924 23:13:23.845897 27722 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:23.846102 27722 heartbeater.cc:511] Master 127.12.212.62:42995 requested a full tablet report, sending...
I20260924 23:13:23.846493 27396 ts_manager.cc:194] Registered new tserver with Master: 4f92be39417149649ce5b36808fbd57f (127.12.212.2:40723)
I20260924 23:13:23.846920 27396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:54401
W20260924 23:13:23.920603 27725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:23.920830 27725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:23.920855 27725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:23.922036 27725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:23.922071 27725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:23.922097 27725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:13:23.923540 27725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:42059
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=45431
--tserver_master_addrs=127.12.212.62:42995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:23.923738 27725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:23.923925 27725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:13:23.929052 27725 server_base.cc:1034] running on GCE node
W20260924 23:13:23.929023 27730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:23.929011 27733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:23.929023 27731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:23.929453 27725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:23.929703 27725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:23.930852 27725 hybrid_clock.cc:648] HybridClock initialized: now 1790291603930837 us; error 34 us; skew 500 ppm
Sep 24 23:13:23 dist-test-slave-m0z6 krb5kdc[26815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291603, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:23.933422 27725 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:13:23.933878 27725 webserver.cc:460] Webserver started at http://127.12.212.3:45431/ using document root <none> and password file <none>
I20260924 23:13:23.934098 27725 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:23.934160 27725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:23.935598 27725 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:23.936201 27740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:23.936358 27725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:13:23.936442 27725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "394b5678af3a476dbc8b8455e39c8be8"
format_stamp: "Formatted at 2026-09-24 23:13:23 on dist-test-slave-m0z6"
server_key: "13e27d3c3198054c4a4fa2912809bf6e"
server_key_iv: "aa6bba64170ebc082e31182c7cf3af81"
server_key_version: "encryptionkey@0"
I20260924 23:13:23.936790 27725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:23.944419 27725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:23.946554 27725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:23.946722 27725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:23.946973 27725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:13:23.947274 27725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:13:23.947307 27725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:23.947336 27725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:13:23.947360 27725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:23.952955 27725 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:42059
I20260924 23:13:23.953027 27853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:42059 every 8 connection(s)
I20260924 23:13:23.953337 27725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
Sep 24 23:13:23 dist-test-slave-m0z6 krb5kdc[26815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291603, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:23.961771 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 27725
I20260924 23:13:23.963369 27587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.954074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:34117 (local address 127.12.212.62:42995)
0924 23:13:23.954151 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:23.954154 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:23.954325 (+ 171us) server_negotiation.cc:407] Connection header received
0924 23:13:23.954907 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:23.954909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:23.954919 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:23.954976 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:23.955958 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.956437 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.957486 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.957692 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.957748 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.960033 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:23.960035 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:23.960035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:23.960043 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:23.961565 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:23.962182 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:23.962184 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:23.962184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:23.962281 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:23.962632 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:23.962633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:23.962634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:23.962827 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:23.962900 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:23.963186 (+ 286us) server_negotiation.cc:299] Negotiation successful
0924 23:13:23.963274 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 23:13:23.963523 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.954048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:34117)
0924 23:13:23.954203 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:23.954242 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:23.954800 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:23.955051 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:23.955055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:23.955674 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:23.955875 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.955880 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.956532 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.956533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.957387 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.957390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.957761 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.957762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.957869 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.958405 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:23.958414 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:23.959908 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:23.961712 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:23.961726 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:23.961728 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:23.962073 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:23.962357 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:23.962359 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:23.962359 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:23.962566 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:23.962890 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:23.962897 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:23.963045 (+ 148us) client_negotiation.cc:769] Sending connection context
0924 23:13:23.963144 (+ 99us) client_negotiation.cc:240] Negotiation successful
0924 23:13:23.963287 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260924 23:13:23.963989 27854 heartbeater.cc:347] Connected to a master server at 127.12.212.62:42995
I20260924 23:13:23.964094 27854 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:23.964309 27854 heartbeater.cc:511] Master 127.12.212.62:42995 requested a full tablet report, sending...
I20260924 23:13:23.964723 27396 ts_manager.cc:194] Registered new tserver with Master: 394b5678af3a476dbc8b8455e39c8be8 (127.12.212.3:42059)
I20260924 23:13:23.965164 27396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:34117
I20260924 23:13:23.969193 27858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.962198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52432 (local address 127.12.212.62:42995)
0924 23:13:23.962332 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:23.962334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:23.962448 (+ 114us) server_negotiation.cc:407] Connection header received
0924 23:13:23.962506 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:23.962507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:23.962519 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:23.962642 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:23.963340 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.964056 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.965018 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.965292 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.965359 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.966685 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:23.966686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:23.966687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:23.966717 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:23.968044 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:23.968439 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:23.968440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:23.968441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:23.968501 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:23.968685 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:23.968686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:23.968686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:23.968803 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:23.968972 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:23.969007 (+ 35us) server_negotiation.cc:299] Negotiation successful
0924 23:13:23.969094 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260924 23:13:23.970021 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 23:13:23.972990 27858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.970413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52442 (local address 127.12.212.62:42995)
0924 23:13:23.970473 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:23.970475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:23.970571 (+ 96us) server_negotiation.cc:407] Connection header received
0924 23:13:23.970635 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:23.970636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:23.970669 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:23.970756 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:13:23.970870 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.971370 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.972214 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.972418 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.972490 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.972652 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:13:23.972765 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:13:23.972835 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:23.972871 (+ 36us) server_negotiation.cc:299] Negotiation successful
0924 23:13:23.972896 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 23:13:23.973333 27396 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:52442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:13:23.975076 27396 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:13:23.977443 27518 tablet_service.cc:1467] Processing CreateTablet for tablet 0e03c063570e4afda540d244c25af9a3 (DEFAULT_TABLE table=test-table [id=dd95fcaa8d2e4adfafc0897eea7f4546]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:13:23.977653 27518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e03c063570e4afda540d244c25af9a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:23.979136 27476 tablet_bootstrap.cc:492] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a: Bootstrap starting.
I20260924 23:13:23.979461 27476 tablet_bootstrap.cc:654] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a: Neither blocks nor log segments found. Creating new log.
I20260924 23:13:23.980295 27476 tablet_bootstrap.cc:492] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a: No bootstrap required, opened a new log
I20260924 23:13:23.980350 27476 ts_tablet_manager.cc:1397] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260924 23:13:23.980490 27476 raft_consensus.cc:348] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } }
I20260924 23:13:23.980574 27476 raft_consensus.cc:374] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:13:23.980650 27476 raft_consensus.cc:729] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: add5c68ef5144558b24747730bf38f0a, State: Initialized, Role: FOLLOWER
I20260924 23:13:23.980712 27476 consensus_queue.cc:260] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } }
I20260924 23:13:23.980815 27476 ts_tablet_manager.cc:1428] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:23.981638 27591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.977020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:42059 (local address 127.0.0.1:55102)
0924 23:13:23.977133 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:23.977143 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:23.977176 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:23.977768 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:23.977770 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:23.977772 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:13:23.977839 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.977841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.979311 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.979312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.980335 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.980338 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.981337 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.981338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.981415 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.981442 (+ 27us) client_negotiation.cc:769] Sending connection context
0924 23:13:23.981494 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 23:13:23.981553 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260924 23:13:23.981640 27862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.977239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55102 (local address 127.12.212.3:42059)
0924 23:13:23.977445 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:23.977451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:23.977462 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:13:23.977504 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:23.977506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:23.977524 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:23.977934 (+ 410us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:13:23.978029 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.979212 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.980497 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.981259 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.981326 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.981392 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:23.981518 (+ 126us) server_negotiation.cc:299] Negotiation successful
0924 23:13:23.981557 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":81,"threads_started":1}
I20260924 23:13:23.982455 27788 tablet_service.cc:1467] Processing CreateTablet for tablet 0e03c063570e4afda540d244c25af9a3 (DEFAULT_TABLE table=test-table [id=dd95fcaa8d2e4adfafc0897eea7f4546]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:13:23.982890 27788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e03c063570e4afda540d244c25af9a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:23.983644 27860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.977052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:40723 (local address 127.0.0.1:42048)
0924 23:13:23.977299 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:23.977308 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:23.977360 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:23.977782 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:23.977783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:23.977786 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:13:23.977848 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.977850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.981278 (+ 3428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.981279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.982331 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.982335 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.983371 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.983372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:23.983433 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.983438 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 23:13:23.983487 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 23:13:23.983526 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":106,"threads_started":1}
I20260924 23:13:23.983656 27861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.977173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42048 (local address 127.12.212.2:40723)
0924 23:13:23.977533 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:23.977537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:23.977546 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:13:23.977583 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:23.977585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:23.977694 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:23.977783 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:13:23.977934 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.981032 (+ 3098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.982440 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.983239 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.983335 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:23.983409 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:23.983525 (+ 116us) server_negotiation.cc:299] Negotiation successful
0924 23:13:23.983577 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":106,"threads_started":1}
I20260924 23:13:23.986502 27864 tablet_bootstrap.cc:492] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: Bootstrap starting.
I20260924 23:13:23.987241 27864 tablet_bootstrap.cc:654] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: Neither blocks nor log segments found. Creating new log.
I20260924 23:13:23.987169 27656 tablet_service.cc:1467] Processing CreateTablet for tablet 0e03c063570e4afda540d244c25af9a3 (DEFAULT_TABLE table=test-table [id=dd95fcaa8d2e4adfafc0897eea7f4546]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:13:23.987488 27656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e03c063570e4afda540d244c25af9a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:23.987731 27864 log.cc:824] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:23.988497 27864 tablet_bootstrap.cc:492] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: No bootstrap required, opened a new log
I20260924 23:13:23.988570 27864 ts_tablet_manager.cc:1397] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:13:23.990217 27864 raft_consensus.cc:348] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } }
I20260924 23:13:23.990401 27864 raft_consensus.cc:374] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:13:23.990540 27864 raft_consensus.cc:729] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 394b5678af3a476dbc8b8455e39c8be8, State: Initialized, Role: FOLLOWER
I20260924 23:13:23.990729 27866 tablet_bootstrap.cc:492] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f: Bootstrap starting.
I20260924 23:13:23.990742 27864 consensus_queue.cc:260] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } }
I20260924 23:13:23.991089 27854 heartbeater.cc:503] Master 127.12.212.62:42995 was elected leader, sending a full tablet report...
I20260924 23:13:23.991091 27864 ts_tablet_manager.cc:1428] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 23:13:23.991426 27866 tablet_bootstrap.cc:654] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f: Neither blocks nor log segments found. Creating new log.
I20260924 23:13:23.992139 27866 log.cc:824] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:23.993041 27866 tablet_bootstrap.cc:492] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f: No bootstrap required, opened a new log
I20260924 23:13:23.993125 27866 ts_tablet_manager.cc:1397] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260924 23:13:23.994681 27866 raft_consensus.cc:348] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } }
I20260924 23:13:23.994900 27866 raft_consensus.cc:374] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:13:23.994936 27866 raft_consensus.cc:729] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f92be39417149649ce5b36808fbd57f, State: Initialized, Role: FOLLOWER
I20260924 23:13:23.995067 27866 consensus_queue.cc:260] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } }
I20260924 23:13:23.995378 27866 ts_tablet_manager.cc:1428] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:13:23.995443 27722 heartbeater.cc:503] Master 127.12.212.62:42995 was elected leader, sending a full tablet report...
I20260924 23:13:23.997377 27869 raft_consensus.cc:482] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:13:23.997493 27869 raft_consensus.cc:504] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } }
I20260924 23:13:23.997850 27869 leader_election.cc:288] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 394b5678af3a476dbc8b8455e39c8be8 (127.12.212.3:42059), add5c68ef5144558b24747730bf38f0a (127.12.212.1:42655)
I20260924 23:13:24.001820 27592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.998316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:37147 (local address 127.12.212.1:42655)
0924 23:13:23.998366 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:23.998368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:23.998376 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:13:23.998398 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:23.998398 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:23.998431 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:23.998494 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:13:23.998616 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.999362 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:24.000813 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:24.001459 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:24.001540 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:24.001637 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:24.001719 (+ 82us) server_negotiation.cc:299] Negotiation successful
0924 23:13:24.001757 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 23:13:24.001849 27870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.998112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:42655 (local address 127.12.212.2:37147)
0924 23:13:23.998254 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:23.998262 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:23.998299 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:23.998508 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:23.998508 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:23.998510 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:13:23.998557 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.998559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.999432 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.999432 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:24.000682 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:24.000687 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:24.001558 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:24.001559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:24.001639 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:24.001645 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 23:13:24.001712 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 23:13:24.001758 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":6,"thread_start_us":95,"threads_started":1}
I20260924 23:13:24.002030 27724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.998005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:42059 (local address 127.12.212.2:59947)
0924 23:13:23.998065 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:23.998076 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:23.998136 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:23.998289 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:23.998290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:23.998293 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:13:23.998368 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:23.998370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.999381 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:23.999382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:24.000682 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:24.000687 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:24.001799 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:24.001800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:24.001871 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:24.001874 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:13:24.001922 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 23:13:24.001963 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":25}
I20260924 23:13:24.002151 27538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e03c063570e4afda540d244c25af9a3" candidate_uuid: "4f92be39417149649ce5b36808fbd57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "add5c68ef5144558b24747730bf38f0a" is_pre_election: true
I20260924 23:13:24.002292 27862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:23.998070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:59947 (local address 127.12.212.3:42059)
0924 23:13:23.998125 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:23.998128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:23.998136 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:13:23.998199 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:23.998201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:23.998209 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:23.998267 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:13:23.998448 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:23.999278 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:24.000812 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:24.001453 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:24.001540 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:24.001550 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:24.002178 (+ 628us) server_negotiation.cc:299] Negotiation successful
0924 23:13:24.002233 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 23:13:24.002334 27538 raft_consensus.cc:2454] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f92be39417149649ce5b36808fbd57f in term 0.
I20260924 23:13:24.002556 27611 leader_election.cc:302] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f92be39417149649ce5b36808fbd57f, add5c68ef5144558b24747730bf38f0a; no voters:
I20260924 23:13:24.002588 27808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e03c063570e4afda540d244c25af9a3" candidate_uuid: "4f92be39417149649ce5b36808fbd57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "394b5678af3a476dbc8b8455e39c8be8" is_pre_election: true
I20260924 23:13:24.002724 27869 raft_consensus.cc:2781] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:13:24.002784 27869 raft_consensus.cc:482] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:13:24.002808 27869 raft_consensus.cc:3037] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:13:24.002802 27808 raft_consensus.cc:2454] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f92be39417149649ce5b36808fbd57f in term 0.
I20260924 23:13:24.003607 27869 raft_consensus.cc:504] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } }
I20260924 23:13:24.003741 27869 leader_election.cc:288] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [CANDIDATE]: Term 1 election: Requested vote from peers 394b5678af3a476dbc8b8455e39c8be8 (127.12.212.3:42059), add5c68ef5144558b24747730bf38f0a (127.12.212.1:42655)
I20260924 23:13:24.003942 27808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e03c063570e4afda540d244c25af9a3" candidate_uuid: "4f92be39417149649ce5b36808fbd57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "394b5678af3a476dbc8b8455e39c8be8"
I20260924 23:13:24.003973 27538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e03c063570e4afda540d244c25af9a3" candidate_uuid: "4f92be39417149649ce5b36808fbd57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "add5c68ef5144558b24747730bf38f0a"
I20260924 23:13:24.004031 27808 raft_consensus.cc:3037] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:13:24.004052 27538 raft_consensus.cc:3037] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:13:24.004822 27808 raft_consensus.cc:2454] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f92be39417149649ce5b36808fbd57f in term 1.
I20260924 23:13:24.004837 27538 raft_consensus.cc:2454] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f92be39417149649ce5b36808fbd57f in term 1.
I20260924 23:13:24.005012 27611 leader_election.cc:302] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f92be39417149649ce5b36808fbd57f, add5c68ef5144558b24747730bf38f0a; no voters:
I20260924 23:13:24.005112 27869 raft_consensus.cc:2781] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:13:24.005298 27869 raft_consensus.cc:686] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [term 1 LEADER]: Becoming Leader. State: Replica: 4f92be39417149649ce5b36808fbd57f, State: Running, Role: LEADER
I20260924 23:13:24.005414 27869 consensus_queue.cc:237] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } }
I20260924 23:13:24.006354 27396 catalog_manager.cc:5495] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f92be39417149649ce5b36808fbd57f (127.12.212.2). New cstate: current_term: 1 leader_uuid: "4f92be39417149649ce5b36808fbd57f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } health_report { overall_health: UNKNOWN } } }
I20260924 23:13:24.018478 27861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:24.016033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42056 (local address 127.12.212.2:40723)
0924 23:13:24.016135 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:24.016137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:24.016146 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:13:24.016182 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:24.016183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:24.016200 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:24.016263 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:13:24.016403 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:24.016970 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:24.017697 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:24.017878 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:24.017923 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:24.018048 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:13:24.018178 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:13:24.018244 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:24.018347 (+ 103us) server_negotiation.cc:299] Negotiation successful
0924 23:13:24.018371 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260924 23:13:24.020563 27538 raft_consensus.cc:1261] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [term 1 FOLLOWER]: Refusing update from remote peer 4f92be39417149649ce5b36808fbd57f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:13:24.020561 27808 raft_consensus.cc:1261] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8 [term 1 FOLLOWER]: Refusing update from remote peer 4f92be39417149649ce5b36808fbd57f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:13:24.020872 27869 consensus_queue.cc:1035] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [LEADER]: Connected to new peer: Peer: permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:13:24.021018 27871 consensus_queue.cc:1035] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [LEADER]: Connected to new peer: Peer: permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:13:24.023778 27875 mvcc.cc:205] Tried to move back new op lower bound from 7333034410065281024 to 7333034410007171072. Current Snapshot: MvccSnapshot[applied={T|T < 7333034410065281024}]
I20260924 23:13:24.025561 27396 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:52442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:13:24.025669 27396 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:52442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:13:24.026368 27396 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec: Sending DeleteTablet for 3 replicas of tablet 0e03c063570e4afda540d244c25af9a3
I20260924 23:13:24.026734 27788 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e03c063570e4afda540d244c25af9a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:13:24 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:55102
I20260924 23:13:24.026782 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 27434
I20260924 23:13:24.026801 27656 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e03c063570e4afda540d244c25af9a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:13:24 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:42048
I20260924 23:13:24.026830 27518 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e03c063570e4afda540d244c25af9a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:13:24 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:42536
I20260924 23:13:24.027441 27882 tablet_replica.cc:330] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: stopping tablet replica
I20260924 23:13:24.027558 27882 raft_consensus.cc:2229] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:13:24.027683 27882 raft_consensus.cc:2258] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:13:24.028151 27882 ts_tablet_manager.cc:1905] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:13:24.029954 27882 ts_tablet_manager.cc:1918] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:13:24.030045 27882 log.cc:1196] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/0e03c063570e4afda540d244c25af9a3
I20260924 23:13:24.030319 27882 ts_tablet_manager.cc:1939] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: Deleting consensus metadata
I20260924 23:13:24.030946 27382 catalog_manager.cc:4841] TS 394b5678af3a476dbc8b8455e39c8be8 (127.12.212.3:42059): tablet 0e03c063570e4afda540d244c25af9a3 (table test-table [id=dd95fcaa8d2e4adfafc0897eea7f4546]) successfully deleted
I20260924 23:13:24.038025 27883 tablet_replica.cc:330] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f: stopping tablet replica
W20260924 23:13:24.038141 27382 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.212.1:42655 (error 108)
I20260924 23:13:24.038192 27883 raft_consensus.cc:2229] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:13:24.038316 27883 raft_consensus.cc:2258] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:13:24.038592 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 27590
I20260924 23:13:24.040378 27860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:24.039573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:42655 (local address 127.0.0.1:42540)
0924 23:13:24.040283 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:24.040336 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.1:42655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":680}
W20260924 23:13:24.040524 27382 catalog_manager.cc:4584] TS add5c68ef5144558b24747730bf38f0a (127.12.212.1:42655): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0e03c063570e4afda540d244c25af9a3: Network error: Client connection negotiation failed: client connection to 127.12.212.1:42655: connect: Connection refused (error 111)
I20260924 23:13:24.044941 27860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:24.044778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:40723 (local address 127.0.0.1:42058)
0924 23:13:24.044856 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:24.044897 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.2:40723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260924 23:13:24.044919 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 27725
I20260924 23:13:24.049736 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 27363
I20260924 23:13:24.054672 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:42995
--webserver_interface=127.12.212.62
--webserver_port=35103
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:42995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:13:24.137703 27885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:24.137923 27885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:24.137945 27885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:24.139151 27885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:13:24.139194 27885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:24.139212 27885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:24.139230 27885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:13:24.139245 27885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:13:24.141256 27885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:42995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:42995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=35103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:24.141544 27885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:24.141837 27885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:24.146981 27893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:24.146973 27890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:24.146973 27891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:24.147132 27885 server_base.cc:1034] running on GCE node
I20260924 23:13:24.147385 27885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:24.147667 27885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:24.148877 27885 hybrid_clock.cc:648] HybridClock initialized: now 1790291604148865 us; error 28 us; skew 500 ppm
Sep 24 23:13:24 dist-test-slave-m0z6 krb5kdc[26815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291604, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:24.151774 27885 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:13:24.152300 27885 webserver.cc:460] Webserver started at http://127.12.212.62:35103/ using document root <none> and password file <none>
I20260924 23:13:24.152546 27885 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:24.152594 27885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:24.153998 27885 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:13:24.154836 27900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:24.155017 27885 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:13:24.155117 27885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "99a72e374aa044f0afbc5774ba21a5ec"
format_stamp: "Formatted at 2026-09-24 23:13:22 on dist-test-slave-m0z6"
server_key: "ecfc3bea0cc73bcc15b9dfb5b5813a80"
server_key_iv: "c13fc02a10790ef7a8f932b0ebf618ef"
server_key_version: "encryptionkey@0"
I20260924 23:13:24.155481 27885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:24.174582 27885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:24.176875 27885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:24.177004 27885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:24.180845 27885 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:42995
I20260924 23:13:24.180892 27952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:42995 every 8 connection(s)
I20260924 23:13:24.181268 27885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:13:24.182979 27953 sys_catalog.cc:263] Verifying existing consensus state
I20260924 23:13:24.183666 27953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec: Bootstrap starting.
I20260924 23:13:24.190246 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 27885
I20260924 23:13:24.190632 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:42655
--local_ip_for_outbound_sockets=127.12.212.1
--tserver_master_addrs=127.12.212.62:42995
--webserver_port=44821
--webserver_interface=127.12.212.1
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 23:13:24.192786 27953 log.cc:824] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:24.195492 27953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 23:13:24.195822 27953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec: Bootstrap complete.
I20260924 23:13:24.197954 27953 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a72e374aa044f0afbc5774ba21a5ec" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42995 } }
I20260924 23:13:24.198205 27953 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a72e374aa044f0afbc5774ba21a5ec, State: Initialized, Role: FOLLOWER
I20260924 23:13:24.198357 27953 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a72e374aa044f0afbc5774ba21a5ec" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42995 } }
I20260924 23:13:24.198426 27953 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:13:24.198472 27953 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:13:24.198525 27953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 2 FOLLOWER]: Advancing to term 3
I20260924 23:13:24.199620 27953 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a72e374aa044f0afbc5774ba21a5ec" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42995 } }
I20260924 23:13:24.199728 27953 leader_election.cc:302] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99a72e374aa044f0afbc5774ba21a5ec; no voters:
I20260924 23:13:24.199908 27953 leader_election.cc:288] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [CANDIDATE]: Term 3 election: Requested vote from peers
I20260924 23:13:24.199967 27957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 3 FOLLOWER]: Leader election won for term 3
I20260924 23:13:24.200089 27957 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [term 3 LEADER]: Becoming Leader. State: Replica: 99a72e374aa044f0afbc5774ba21a5ec, State: Running, Role: LEADER
I20260924 23:13:24.200191 27953 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:13:24.200183 27957 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a72e374aa044f0afbc5774ba21a5ec" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42995 } }
I20260924 23:13:24.200588 27957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99a72e374aa044f0afbc5774ba21a5ec. Latest consensus state: current_term: 3 leader_uuid: "99a72e374aa044f0afbc5774ba21a5ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a72e374aa044f0afbc5774ba21a5ec" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42995 } } }
I20260924 23:13:24.200665 27957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [sys.catalog]: This master's current role is: LEADER
I20260924 23:13:24.200794 27958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "99a72e374aa044f0afbc5774ba21a5ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a72e374aa044f0afbc5774ba21a5ec" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42995 } } }
I20260924 23:13:24.200857 27958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99a72e374aa044f0afbc5774ba21a5ec [sys.catalog]: This master's current role is: LEADER
I20260924 23:13:24.201210 27962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:13:24.202013 27962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:13:24.202133 27962 catalog_manager.cc:1259] Loaded cluster ID: c182d3ed9e2346ec920bba11a8df7737
I20260924 23:13:24.202366 27962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260924 23:13:24.202586 27962 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd6cddf49ad google::LogMessage::Fail() at ??:0
@ 0x7fd6cddf788a google::LogMessage::SendToLog() at ??:0
@ 0x7fd6cddf44de google::LogMessage::Flush() at ??:0
@ 0x7fd6cddf5a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd6d5d150f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fd6d5d1e8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd6cfcc20d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd6cfcb7954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd6d0b1b6db start_thread at ??:0
@ 0x7fd6cc8f261f clone at ??:0
W20260924 23:13:24.290853 27955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:24.291091 27955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:24.291122 27955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:24.292834 27955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:24.292912 27955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:24.292945 27955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:13:24.294868 27955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:42655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=44821
--tserver_master_addrs=127.12.212.62:42995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:24.295145 27955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:24.295398 27955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:24.300772 27989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:24.300815 27955 server_base.cc:1034] running on GCE node
W20260924 23:13:24.300901 27990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:24.300765 27992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:24.301227 27955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:24.301518 27955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:24.302774 27955 hybrid_clock.cc:648] HybridClock initialized: now 1790291604302755 us; error 29 us; skew 500 ppm
Sep 24 23:13:24 dist-test-slave-m0z6 krb5kdc[26815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291604, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:24.305655 27955 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:13:24.306208 27955 webserver.cc:460] Webserver started at http://127.12.212.1:44821/ using document root <none> and password file <none>
I20260924 23:13:24.306437 27955 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:24.306486 27955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:24.307993 27955 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:24.308759 27999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:24.308955 27955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:24.309031 27955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "add5c68ef5144558b24747730bf38f0a"
format_stamp: "Formatted at 2026-09-24 23:13:22 on dist-test-slave-m0z6"
server_key: "2a1ae48a1c164fa6c42908a34570ad88"
server_key_iv: "6524ac436028fd6a19396c10e27244a0"
server_key_version: "encryptionkey@0"
I20260924 23:13:24.309398 27955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:24.332140 27955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:24.334473 27955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:24.334617 27955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:24.334895 27955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:13:24.335386 28006 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 23:13:24.337065 27955 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 23:13:24.337123 27955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 23:13:24.337152 27955 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 23:13:24.337903 27955 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 23:13:24.337947 27955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 23:13:24.338028 28006 tablet_bootstrap.cc:492] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a: Bootstrap starting.
I20260924 23:13:24.345968 27955 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:42655
I20260924 23:13:24.346551 27955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:13:24.346836 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 27955
I20260924 23:13:24.347117 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:40723
--local_ip_for_outbound_sockets=127.12.212.2
--tserver_master_addrs=127.12.212.62:42995
--webserver_port=37661
--webserver_interface=127.12.212.2
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 23:13:24.347167 28113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:42655 every 8 connection(s)
I20260924 23:13:24.349692 28006 log.cc:824] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:24.352734 28006 tablet_bootstrap.cc:492] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 23:13:24.353202 28006 tablet_bootstrap.cc:492] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a: Bootstrap complete.
I20260924 23:13:24.354394 28006 ts_tablet_manager.cc:1397] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.009s
I20260924 23:13:24.356304 28006 raft_consensus.cc:348] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } }
I20260924 23:13:24.356702 28006 raft_consensus.cc:729] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: add5c68ef5144558b24747730bf38f0a, State: Initialized, Role: FOLLOWER
I20260924 23:13:24.356905 28006 consensus_queue.cc:260] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } }
I20260924 23:13:24.357247 28006 ts_tablet_manager.cc:1428] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a: Time spent starting tablet: real 0.003s user 0.003s sys 0.003s
W20260924 23:13:24.409302 28116 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:13:24.347835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:59741)
0924 23:13:24.348057 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:24.348114 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:24.348823 (+ 709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:24.409102 (+ 60279us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":126,"threads_started":1}
W20260924 23:13:24.409587 28114 heartbeater.cc:644] Failed to heartbeat to 127.12.212.62:42995 (0 consecutive failures): Network error: Failed to ping master at 127.12.212.62:42995: Client connection negotiation failed: client connection to 127.12.212.62:42995: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 23:13:24.409924 28116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:24.409815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:40463)
0924 23:13:24.409863 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:24.409903 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260924 23:13:24.410329 28116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:24.410197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:48919)
0924 23:13:24.410264 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:24.410303 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260924 23:13:24.410459 28114 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 23:13:24.436189 28117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:24.436453 28117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:24.436482 28117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:24.437997 28117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:24.438072 28117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:24.438107 28117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:13:24.439951 28117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:40723
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=37661
--tserver_master_addrs=127.12.212.62:42995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:24.440244 28117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:24.440500 28117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:24.445696 28125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:24.445771 28124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:24.445807 28127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:24.446053 28117 server_base.cc:1034] running on GCE node
I20260924 23:13:24.446305 28117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:24.446552 28117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:24.447717 28117 hybrid_clock.cc:648] HybridClock initialized: now 1790291604447703 us; error 33 us; skew 500 ppm
Sep 24 23:13:24 dist-test-slave-m0z6 krb5kdc[26815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291604, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:24.450417 28117 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:13:24.450959 28117 webserver.cc:460] Webserver started at http://127.12.212.2:37661/ using document root <none> and password file <none>
I20260924 23:13:24.451152 28117 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:24.451197 28117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:24.452646 28117 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:24.453472 28134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:24.453686 28117 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:24.453778 28117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "4f92be39417149649ce5b36808fbd57f"
format_stamp: "Formatted at 2026-09-24 23:13:23 on dist-test-slave-m0z6"
server_key: "aa402f4d7e9a877788c8acc59299e50f"
server_key_iv: "b6e53b246584c718f16b9ec3149d0291"
server_key_version: "encryptionkey@0"
I20260924 23:13:24.454169 28117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:24.470607 28117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:24.472925 28117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:24.473049 28117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:24.473253 28117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:13:24.473656 28141 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 23:13:24.475209 28117 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 23:13:24.475273 28117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 23:13:24.475314 28117 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 23:13:24.476027 28117 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 23:13:24.476060 28117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 23:13:24.476157 28141 tablet_bootstrap.cc:492] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f: Bootstrap starting.
I20260924 23:13:24.482021 28117 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:40723
I20260924 23:13:24.482129 28248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:40723 every 8 connection(s)
I20260924 23:13:24.482452 28117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:13:24.483673 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:24.483147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:56015)
0924 23:13:24.483338 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:24.483457 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":73,"threads_started":1}
I20260924 23:13:24.483702 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 28117
I20260924 23:13:24.483919 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:42059
--local_ip_for_outbound_sockets=127.12.212.3
--tserver_master_addrs=127.12.212.62:42995
--webserver_port=45431
--webserver_interface=127.12.212.3
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:13:24.485160 28249 heartbeater.cc:644] Failed to heartbeat to 127.12.212.62:42995 (0 consecutive failures): Network error: Failed to ping master at 127.12.212.62:42995: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
I20260924 23:13:24.485507 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:24.485402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:58383)
0924 23:13:24.485453 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:24.485482 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260924 23:13:24.485838 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:24.485750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:50515)
0924 23:13:24.485790 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:24.485814 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260924 23:13:24.485924 28249 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 23:13:24.486604 28141 log.cc:824] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:24.488797 28141 tablet_bootstrap.cc:492] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 23:13:24.489054 28141 tablet_bootstrap.cc:492] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f: Bootstrap complete.
I20260924 23:13:24.489828 28141 ts_tablet_manager.cc:1397] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.006s
I20260924 23:13:24.491242 28141 raft_consensus.cc:348] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } }
I20260924 23:13:24.491514 28141 raft_consensus.cc:729] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f92be39417149649ce5b36808fbd57f, State: Initialized, Role: FOLLOWER
I20260924 23:13:24.491640 28141 consensus_queue.cc:260] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } }
I20260924 23:13:24.491873 28141 ts_tablet_manager.cc:1428] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f: Time spent starting tablet: real 0.002s user 0.004s sys 0.001s
I20260924 23:13:24.492197 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:24.492089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:43153)
0924 23:13:24.492144 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:24.492173 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 23:13:24.492514 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:24.492391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:40419)
0924 23:13:24.492440 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:24.492484 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260924 23:13:24.568779 28252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:24.569005 28252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:24.569033 28252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:24.570338 28252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:24.570395 28252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:24.570425 28252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:13:24.572075 28252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:42059
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=45431
--tserver_master_addrs=127.12.212.62:42995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:24.572364 28252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:24.572606 28252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:24.577757 28262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:24.577757 28260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:24.577862 28252 server_base.cc:1034] running on GCE node
W20260924 23:13:24.577955 28259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:24.578168 28252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:24.578418 28252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:24.579568 28252 hybrid_clock.cc:648] HybridClock initialized: now 1790291604579553 us; error 32 us; skew 500 ppm
Sep 24 23:13:24 dist-test-slave-m0z6 krb5kdc[26815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291604, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:24.582163 28252 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:13:24.582655 28252 webserver.cc:460] Webserver started at http://127.12.212.3:45431/ using document root <none> and password file <none>
I20260924 23:13:24.582906 28252 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:24.582957 28252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:24.584408 28252 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:13:24.585196 28269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:24.585383 28252 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:13:24.585477 28252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "394b5678af3a476dbc8b8455e39c8be8"
format_stamp: "Formatted at 2026-09-24 23:13:23 on dist-test-slave-m0z6"
server_key: "13e27d3c3198054c4a4fa2912809bf6e"
server_key_iv: "aa6bba64170ebc082e31182c7cf3af81"
server_key_version: "encryptionkey@0"
I20260924 23:13:24.585878 28252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:24.603816 28252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:24.606019 28252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:24.606158 28252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:24.606387 28252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:13:24.606725 28252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:13:24.606770 28252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:24.606801 28252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:13:24.606823 28252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:24.612365 28252 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:42059
I20260924 23:13:24.612440 28382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:42059 every 8 connection(s)
I20260924 23:13:24.612784 28252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:13:24.613835 28385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:24.613457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:51373)
0924 23:13:24.613608 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:24.613688 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
W20260924 23:13:24.614223 28383 heartbeater.cc:644] Failed to heartbeat to 127.12.212.62:42995 (0 consecutive failures): Network error: Failed to ping master at 127.12.212.62:42995: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
I20260924 23:13:24.614460 28385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:24.614378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:32833)
0924 23:13:24.614417 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:24.614442 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260924 23:13:24.614799 28385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:24.614736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:59957)
0924 23:13:24.614763 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:24.614784 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260924 23:13:24.614892 28383 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 23:13:24.619429 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 28252
I20260924 23:13:25.411262 28387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:25.410982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:57513)
0924 23:13:25.411142 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:25.411191 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":69,"threads_started":1}
I20260924 23:13:25.493367 28388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:25.493091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:52283)
0924 23:13:25.493255 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:25.493304 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":66,"threads_started":1}
I20260924 23:13:25.615510 28389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:25.615295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:54235)
0924 23:13:25.615409 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:25.615452 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":60,"threads_started":1}
I20260924 23:13:25.979481 28390 raft_consensus.cc:482] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:13:25.979633 28390 raft_consensus.cc:504] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } }
I20260924 23:13:25.979988 28390 leader_election.cc:288] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f92be39417149649ce5b36808fbd57f (127.12.212.2:40723), 394b5678af3a476dbc8b8455e39c8be8 (127.12.212.3:42059)
Sep 24 23:13:25 dist-test-slave-m0z6 krb5kdc[26815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291604, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.3@KRBTEST.COM
Sep 24 23:13:25 dist-test-slave-m0z6 krb5kdc[26815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291604, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.2@KRBTEST.COM
I20260924 23:13:25.992830 28394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:25.980309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:59069 (local address 127.12.212.2:40723)
0924 23:13:25.980470 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:25.980477 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:25.980494 (+ 17us) server_negotiation.cc:407] Connection header received
0924 23:13:25.980607 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:25.980610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:25.980660 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:25.980795 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:25.981711 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:25.983210 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:25.984745 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:25.985416 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:25.985539 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:25.989001 (+ 3462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:25.989007 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:25.989007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:25.989022 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:25.990923 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:25.991387 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:25.991391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:25.991391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:25.991490 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:25.991778 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:25.991779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:25.991779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:25.992104 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:25.992190 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:25.992540 (+ 350us) server_negotiation.cc:299] Negotiation successful
0924 23:13:25.992704 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":50,"threads_started":1}
I20260924 23:13:25.992856 28393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:25.980260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:55313 (local address 127.12.212.3:42059)
0924 23:13:25.980419 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:25.980431 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:25.980465 (+ 34us) server_negotiation.cc:407] Connection header received
0924 23:13:25.981050 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:25.981053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:25.981097 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:25.981203 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:25.981719 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:25.983209 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:25.984742 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:25.985416 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:25.985539 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:25.988556 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:25.988563 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:25.988563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:25.988578 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:25.990923 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:25.991711 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:25.991714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:25.991715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:25.991838 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:25.992049 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:25.992049 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:25.992050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:25.992368 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:25.992427 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:25.992629 (+ 202us) server_negotiation.cc:299] Negotiation successful
0924 23:13:25.992754 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":63,"threads_started":1}
I20260924 23:13:25.992880 28392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:25.980152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:42059 (local address 127.12.212.1:55313)
0924 23:13:25.980301 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:25.980310 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:25.980363 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:25.981223 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:25.981225 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:25.981393 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:25.981577 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:25.981585 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:25.983334 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:25.983334 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:25.984639 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:25.984644 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:25.985547 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:25.985549 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:25.985719 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:25.986384 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:25.986401 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:25.988396 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:25.991371 (+ 2975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:25.991371 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:25.991372 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:25.991583 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:25.991922 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:25.991922 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:25.991923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:25.991986 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:25.992455 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:25.992456 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:25.992493 (+ 37us) client_negotiation.cc:769] Sending connection context
0924 23:13:25.992564 (+ 71us) client_negotiation.cc:240] Negotiation successful
0924 23:13:25.992746 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":110,"thread_start_us":91,"threads_started":1}
I20260924 23:13:25.993038 28391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:25.980125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:40723 (local address 127.12.212.1:59069)
0924 23:13:25.980284 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:25.980294 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:25.980353 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:25.980876 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:25.980879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:25.981277 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:25.981577 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:25.981585 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:25.983330 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:25.983332 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:25.984639 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:25.984644 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:25.985549 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:25.985550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:25.985719 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:25.986406 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:25.986410 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:25.988868 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:25.991034 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:25.991042 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:25.991045 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:25.991293 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:25.991581 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:25.991582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:25.991582 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:25.991714 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:25.992196 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:25.992203 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:25.992348 (+ 145us) client_negotiation.cc:769] Sending connection context
0924 23:13:25.992471 (+ 123us) client_negotiation.cc:240] Negotiation successful
0924 23:13:25.992818 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":146,"thread_start_us":53,"threads_started":1}
W20260924 23:13:25.993867 28002 leader_election.cc:341] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 394b5678af3a476dbc8b8455e39c8be8 (127.12.212.3:42059): Not found: Tablet not found: 0e03c063570e4afda540d244c25af9a3
I20260924 23:13:25.993929 28203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e03c063570e4afda540d244c25af9a3" candidate_uuid: "add5c68ef5144558b24747730bf38f0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4f92be39417149649ce5b36808fbd57f" is_pre_election: true
I20260924 23:13:25.994092 28203 raft_consensus.cc:2454] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate add5c68ef5144558b24747730bf38f0a in term 1.
I20260924 23:13:25.994334 28003 leader_election.cc:302] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f92be39417149649ce5b36808fbd57f, add5c68ef5144558b24747730bf38f0a; no voters: 394b5678af3a476dbc8b8455e39c8be8
I20260924 23:13:25.994451 28390 raft_consensus.cc:2781] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 23:13:25.994509 28390 raft_consensus.cc:482] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:13:25.994536 28390 raft_consensus.cc:3037] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [term 1 FOLLOWER]: Advancing to term 2
I20260924 23:13:25.995746 28390 raft_consensus.cc:504] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } }
I20260924 23:13:25.995877 28390 leader_election.cc:288] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [CANDIDATE]: Term 2 election: Requested vote from peers 4f92be39417149649ce5b36808fbd57f (127.12.212.2:40723), 394b5678af3a476dbc8b8455e39c8be8 (127.12.212.3:42059)
I20260924 23:13:25.996060 28203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e03c063570e4afda540d244c25af9a3" candidate_uuid: "add5c68ef5144558b24747730bf38f0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4f92be39417149649ce5b36808fbd57f"
I20260924 23:13:25.996140 28203 raft_consensus.cc:3037] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [term 1 FOLLOWER]: Advancing to term 2
W20260924 23:13:25.996217 28002 leader_election.cc:341] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 394b5678af3a476dbc8b8455e39c8be8 (127.12.212.3:42059): Not found: Tablet not found: 0e03c063570e4afda540d244c25af9a3
I20260924 23:13:25.997294 28203 raft_consensus.cc:2454] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate add5c68ef5144558b24747730bf38f0a in term 2.
I20260924 23:13:25.997552 28003 leader_election.cc:302] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f92be39417149649ce5b36808fbd57f, add5c68ef5144558b24747730bf38f0a; no voters: 394b5678af3a476dbc8b8455e39c8be8
I20260924 23:13:25.997689 28390 raft_consensus.cc:2781] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [term 2 FOLLOWER]: Leader election won for term 2
I20260924 23:13:25.997848 28390 raft_consensus.cc:686] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [term 2 LEADER]: Becoming Leader. State: Replica: add5c68ef5144558b24747730bf38f0a, State: Running, Role: LEADER
I20260924 23:13:25.997943 28390 consensus_queue.cc:237] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } }
I20260924 23:13:25.998239 28391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:25.998122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:46317)
0924 23:13:25.998175 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:25.998207 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 23:13:26.418334 28203 raft_consensus.cc:1261] T 0e03c063570e4afda540d244c25af9a3 P 4f92be39417149649ce5b36808fbd57f [term 2 FOLLOWER]: Refusing update from remote peer add5c68ef5144558b24747730bf38f0a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260924 23:13:26.418763 28390 consensus_queue.cc:1035] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 23:13:26.419706 28399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:26.419543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:37483)
0924 23:13:26.419618 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:26.419670 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61,"thread_start_us":42,"threads_started":1}
I20260924 23:13:26.420746 28391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:26.420523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:60125)
0924 23:13:26.420665 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:26.420707 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
W20260924 23:13:26.420804 28002 consensus_peers.cc:487] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a -> Peer 394b5678af3a476dbc8b8455e39c8be8 (127.12.212.3:42059): Couldn't send request to peer 394b5678af3a476dbc8b8455e39c8be8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e03c063570e4afda540d244c25af9a3. This is attempt 1: this message will repeat every 5th retry.
I20260924 23:13:26.420955 28399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:26.420817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:43513)
0924 23:13:26.420897 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:26.420927 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260924 23:13:26.616297 28402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:26.616018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:44057)
0924 23:13:26.616202 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:26.616249 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":67,"threads_started":1}
I20260924 23:13:26.919557 28403 ts_tablet_manager.cc:927] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: Initiating tablet copy from peer add5c68ef5144558b24747730bf38f0a (127.12.212.1:42655)
I20260924 23:13:26.919852 28403 tablet_copy_client.cc:316] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: tablet copy: Beginning tablet copy session from remote peer at address 127.12.212.1:42655
Sep 24 23:13:26 dist-test-slave-m0z6 krb5kdc[26815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291604, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.1@KRBTEST.COM
I20260924 23:13:26.928901 28402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:26.920138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:42655 (local address 127.12.212.3:40457)
0924 23:13:26.920229 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:26.920256 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:26.920340 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:26.920694 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:26.920701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:26.921015 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:26.921109 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:26.921111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:26.922017 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:26.922018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:26.922670 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:26.922672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:26.923199 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:26.923200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:26.923280 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:26.923809 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:26.923813 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:26.925202 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:26.927501 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:26.927512 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:26.927515 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:26.927776 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:26.928046 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:26.928046 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:26.928047 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:26.928116 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:26.928520 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:26.928530 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:26.928594 (+ 64us) client_negotiation.cc:769] Sending connection context
0924 23:13:26.928680 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 23:13:26.928796 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260924 23:13:26.928967 28405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:26.920276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:40457 (local address 127.12.212.1:42655)
0924 23:13:26.920448 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:26.920454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:26.920467 (+ 13us) server_negotiation.cc:407] Connection header received
0924 23:13:26.920499 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:26.920501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:26.920518 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:26.920657 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:26.921191 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:26.921918 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:26.922784 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:26.923103 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:26.923170 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:26.925317 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:26.925330 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:26.925330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:26.925338 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:26.927390 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:26.927884 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:26.927888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:26.927888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:26.927961 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:26.928202 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:26.928204 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:26.928204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:26.928396 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:26.928510 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:26.928803 (+ 293us) server_negotiation.cc:299] Negotiation successful
0924 23:13:26.928888 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":75,"threads_started":1}
I20260924 23:13:26.929386 28088 tablet_copy_service.cc:140] P add5c68ef5144558b24747730bf38f0a: Received BeginTabletCopySession request for tablet 0e03c063570e4afda540d244c25af9a3 from peer 394b5678af3a476dbc8b8455e39c8be8 ({username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:40457)
I20260924 23:13:26.929462 28088 tablet_copy_service.cc:161] P add5c68ef5144558b24747730bf38f0a: Beginning new tablet copy session on tablet 0e03c063570e4afda540d244c25af9a3 from peer 394b5678af3a476dbc8b8455e39c8be8 at {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:40457: session id = 394b5678af3a476dbc8b8455e39c8be8-0e03c063570e4afda540d244c25af9a3
I20260924 23:13:26.929956 28088 tablet_copy_source_session.cc:205] T 0e03c063570e4afda540d244c25af9a3 P add5c68ef5144558b24747730bf38f0a: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 23:13:26.931109 28403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e03c063570e4afda540d244c25af9a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:26.934037 28403 tablet_copy_client.cc:791] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: tablet copy: Starting download of 0 data blocks...
I20260924 23:13:26.934194 28403 tablet_copy_client.cc:655] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: tablet copy: Starting download of 1 WAL segments...
I20260924 23:13:26.935123 28403 tablet_copy_client.cc:531] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 23:13:26.935889 28403 tablet_bootstrap.cc:492] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: Bootstrap starting.
I20260924 23:13:26.937260 28403 log.cc:824] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:26.940074 28403 tablet_bootstrap.cc:492] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 23:13:26.940265 28403 tablet_bootstrap.cc:492] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: Bootstrap complete.
I20260924 23:13:26.940372 28403 ts_tablet_manager.cc:1397] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260924 23:13:26.941421 28403 raft_consensus.cc:348] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } }
I20260924 23:13:26.941582 28403 raft_consensus.cc:729] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 394b5678af3a476dbc8b8455e39c8be8, State: Initialized, Role: FOLLOWER
I20260924 23:13:26.941702 28403 consensus_queue.cc:260] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f92be39417149649ce5b36808fbd57f" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 40723 } } peers { permanent_uuid: "394b5678af3a476dbc8b8455e39c8be8" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42059 } } peers { permanent_uuid: "add5c68ef5144558b24747730bf38f0a" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 42655 } }
I20260924 23:13:26.941967 28403 ts_tablet_manager.cc:1428] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 23:13:26.942363 28088 tablet_copy_service.cc:342] P add5c68ef5144558b24747730bf38f0a: Request end of tablet copy session 394b5678af3a476dbc8b8455e39c8be8-0e03c063570e4afda540d244c25af9a3 received from {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:40457
I20260924 23:13:26.942438 28088 tablet_copy_service.cc:433] P add5c68ef5144558b24747730bf38f0a: ending tablet copy session 394b5678af3a476dbc8b8455e39c8be8-0e03c063570e4afda540d244c25af9a3 on tablet 0e03c063570e4afda540d244c25af9a3 with peer 394b5678af3a476dbc8b8455e39c8be8
I20260924 23:13:26.942430 28402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:26.942214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:50339)
0924 23:13:26.942353 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:26.942398 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260924 23:13:27.417161 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:27.416967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:47787)
0924 23:13:27.417077 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:27.417107 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260924 23:13:27.417155 28337 raft_consensus.cc:1203] T 0e03c063570e4afda540d244c25af9a3 P 394b5678af3a476dbc8b8455e39c8be8 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260924 23:13:27.417622 28402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:27.417509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:47445)
0924 23:13:27.417556 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:27.417589 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 23:13:27.421638 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:27.421381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:54097)
0924 23:13:27.421557 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:27.421597 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":50,"threads_started":1}
I20260924 23:13:28.417840 28411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:28.417592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:59599)
0924 23:13:28.417742 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:28.417780 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":58,"threads_started":1}
I20260924 23:13:28.418150 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:28.417976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:41571)
0924 23:13:28.418081 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:28.418111 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":48,"threads_started":1}
I20260924 23:13:28.422317 28413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:28.422096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:41807)
0924 23:13:28.422231 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:28.422272 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":53,"threads_started":1}
I20260924 23:13:29.418641 28415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:29.418428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:38661)
0924 23:13:29.418546 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:29.418584 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":61,"threads_started":1}
I20260924 23:13:29.418853 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:29.418554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:59761)
0924 23:13:29.418756 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:29.418798 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":78,"threads_started":1}
I20260924 23:13:29.419152 28415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:29.419063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:60347)
0924 23:13:29.419106 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:29.419130 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260924 23:13:29.423195 28417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:29.422961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:45211)
0924 23:13:29.423101 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:29.423139 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":59,"threads_started":1}
I20260924 23:13:30.419729 28418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:30.419421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:48597)
0924 23:13:30.419563 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:30.419618 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260924 23:13:30.419909 28419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:30.419645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:42613)
0924 23:13:30.419816 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:30.419858 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":63,"threads_started":1}
I20260924 23:13:30.423916 28420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:30.423696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:60689)
0924 23:13:30.423833 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:30.423868 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":60,"threads_started":1}
I20260924 23:13:31.420413 28422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:31.420185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:53349)
0924 23:13:31.420333 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:31.420367 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":75,"threads_started":1}
I20260924 23:13:31.420495 28423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:31.420292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:38813)
0924 23:13:31.420415 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:31.420447 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260924 23:13:31.424528 28424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:31.424354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:34813)
0924 23:13:31.424464 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:31.424489 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":60,"threads_started":1}
I20260924 23:13:32.421171 28426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:32.420949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:32897)
0924 23:13:32.421084 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:32.421120 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
I20260924 23:13:32.421190 28427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:32.420950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:36261)
0924 23:13:32.421084 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:32.421121 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
I20260924 23:13:32.425150 28428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:32.424972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:49625)
0924 23:13:32.425087 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:32.425111 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
I20260924 23:13:33.421904 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:33.421678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:43869)
0924 23:13:33.421833 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:33.421863 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":73,"threads_started":1}
I20260924 23:13:33.421904 28429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:33.421678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:35429)
0924 23:13:33.421814 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:33.421853 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260924 23:13:33.425896 28431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:33.425624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:38783)
0924 23:13:33.425806 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:33.425851 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":59,"threads_started":1}
I20260924 23:13:34.422619 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:34.422384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:51233)
0924 23:13:34.422541 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:34.422580 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":63,"threads_started":1}
I20260924 23:13:34.422610 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:34.422397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:50519)
0924 23:13:34.422519 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:34.422558 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
I20260924 23:13:34.426680 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:34.426446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:53393)
0924 23:13:34.426589 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:34.426633 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":64,"threads_started":1}
I20260924 23:13:35.423344 28436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:35.423116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:50605)
0924 23:13:35.423248 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:35.423289 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260924 23:13:35.423344 28437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:35.423113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:55333)
0924 23:13:35.423264 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:35.423305 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":68,"threads_started":1}
I20260924 23:13:35.427435 28438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:35.427201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:56811)
0924 23:13:35.427346 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:35.427390 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":73,"threads_started":1}
I20260924 23:13:36.424067 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:36.423832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:48585)
0924 23:13:36.423972 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:36.424015 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":58,"threads_started":1}
I20260924 23:13:36.424067 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:36.423837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:38923)
0924 23:13:36.423976 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:36.424015 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":56,"threads_started":1}
I20260924 23:13:36.428097 28442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:36.427911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:40451)
0924 23:13:36.428024 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:36.428054 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260924 23:13:37.424887 28443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:37.424612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:49663)
0924 23:13:37.424799 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:37.424840 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":134,"threads_started":1}
I20260924 23:13:37.424904 28444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:37.424607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:43985)
0924 23:13:37.424822 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:37.424857 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":139,"threads_started":1}
I20260924 23:13:37.428956 28445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:37.428608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:39609)
0924 23:13:37.428831 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:37.428908 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":136,"threads_started":1}
I20260924 23:13:38.425642 28447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:38.425404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:49417)
0924 23:13:38.425568 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:38.425597 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":78,"threads_started":1}
I20260924 23:13:38.425654 28446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:38.425410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:48157)
0924 23:13:38.425571 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:38.425612 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":71,"threads_started":1}
I20260924 23:13:38.429738 28448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:38.429505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:39067)
0924 23:13:38.429646 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:38.429691 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":58,"threads_started":1}
I20260924 23:13:39.426510 28451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:39.426235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.3:47807)
0924 23:13:39.426424 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:39.426465 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":86,"threads_started":1}
I20260924 23:13:39.426527 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:39.426267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.1:57557)
0924 23:13:39.426419 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:39.426478 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":64,"threads_started":1}
I20260924 23:13:39.430681 28453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:39.430458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42995 (local address 127.12.212.2:51261)
0924 23:13:39.430598 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:39.430633 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
I20260924 23:13:39.622473 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 27955
I20260924 23:13:39.627046 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 28117
I20260924 23:13:39.631382 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 28252
2026-09-24T23:13:39Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18118 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:13:39 dist-test-slave-m0z6 krb5kdc[28456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:13:39 dist-test-slave-m0z6 krb5kdc[28456](info): set up 2 sockets
Sep 24 23:13:39 dist-test-slave-m0z6 krb5kdc[28456](info): commencing operation
krb5kdc: starting...
W20260924 23:13:40.602567 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.952s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:13:40 dist-test-slave-m0z6 krb5kdc[28456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:13:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:13:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:13:40.727123 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:36023
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:36023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:13:40.807299 28469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:40.807544 28469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:40.807581 28469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:40.808903 28469 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 23:13:40.808964 28469 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 23:13:40.808980 28469 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 23:13:40.808996 28469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:13:40.809015 28469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:40.809031 28469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:40.809048 28469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:13:40.809067 28469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:13:40.810865 28469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:36023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:36023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:40.811157 28469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:40.811388 28469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:40.816192 28475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:40.816176 28474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:40.816223 28469 server_base.cc:1034] running on GCE node
W20260924 23:13:40.816176 28477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:40.816702 28469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:40.817013 28469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:40.818156 28469 hybrid_clock.cc:648] HybridClock initialized: now 1790291620818144 us; error 41 us; skew 500 ppm
Sep 24 23:13:40 dist-test-slave-m0z6 krb5kdc[28456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291620, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:40.820577 28469 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:13:40.821050 28469 webserver.cc:460] Webserver started at http://127.12.212.62:35835/ using document root <none> and password file <none>
I20260924 23:13:40.821271 28469 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:40.821321 28469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:40.821449 28469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:13:40.822556 28469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "032c6e3e5d754be1814dc735864864dc"
format_stamp: "Formatted at 2026-09-24 23:13:40 on dist-test-slave-m0z6"
server_key: "6f4b5b4eb1f2b12c8ce70ca717554283"
server_key_iv: "8db77c7fb2ce2aedbc222f834e64f712"
server_key_version: "encryptionkey@0"
I20260924 23:13:40.822933 28469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "032c6e3e5d754be1814dc735864864dc"
format_stamp: "Formatted at 2026-09-24 23:13:40 on dist-test-slave-m0z6"
server_key: "6f4b5b4eb1f2b12c8ce70ca717554283"
server_key_iv: "8db77c7fb2ce2aedbc222f834e64f712"
server_key_version: "encryptionkey@0"
I20260924 23:13:40.824255 28469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:40.825021 28484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:40.825166 28469 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:13:40.825259 28469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "032c6e3e5d754be1814dc735864864dc"
format_stamp: "Formatted at 2026-09-24 23:13:40 on dist-test-slave-m0z6"
server_key: "6f4b5b4eb1f2b12c8ce70ca717554283"
server_key_iv: "8db77c7fb2ce2aedbc222f834e64f712"
server_key_version: "encryptionkey@0"
I20260924 23:13:40.825320 28469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:40.840598 28469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:40.842666 28469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:40.842841 28469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:40.846825 28469 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:36023
I20260924 23:13:40.846899 28536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:36023 every 8 connection(s)
I20260924 23:13:40.847236 28469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:13:40.848097 28537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:40.850561 28537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc: Bootstrap starting.
I20260924 23:13:40.851202 28537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc: Neither blocks nor log segments found. Creating new log.
I20260924 23:13:40.851497 28537 log.cc:824] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:40.852320 28537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc: No bootstrap required, opened a new log
I20260924 23:13:40.852313 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 28469
I20260924 23:13:40.852447 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:13:40.852638 13136 external_mini_cluster.cc:1428] Setting key 456171649bd89b06a6cd268d3d7f68a9
I20260924 23:13:40.854458 28537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032c6e3e5d754be1814dc735864864dc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36023 } }
I20260924 23:13:40.854606 28537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:13:40.854637 28537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 032c6e3e5d754be1814dc735864864dc, State: Initialized, Role: FOLLOWER
I20260924 23:13:40.854846 28537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032c6e3e5d754be1814dc735864864dc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36023 } }
I20260924 23:13:40.854933 28537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:13:40.854960 28537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:13:40.855010 28537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:13:40.855901 28537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032c6e3e5d754be1814dc735864864dc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36023 } }
I20260924 23:13:40.856009 28537 leader_election.cc:302] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 032c6e3e5d754be1814dc735864864dc; no voters:
I20260924 23:13:40.856178 28537 leader_election.cc:288] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:13:40.856232 28542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:13:40.856410 28542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 1 LEADER]: Becoming Leader. State: Replica: 032c6e3e5d754be1814dc735864864dc, State: Running, Role: LEADER
I20260924 23:13:40.856458 28537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:13:40.856498 28542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032c6e3e5d754be1814dc735864864dc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36023 } }
I20260924 23:13:40.857496 28544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 032c6e3e5d754be1814dc735864864dc. Latest consensus state: current_term: 1 leader_uuid: "032c6e3e5d754be1814dc735864864dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032c6e3e5d754be1814dc735864864dc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36023 } } }
I20260924 23:13:40.857589 28544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [sys.catalog]: This master's current role is: LEADER
I20260924 23:13:40.857756 28543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "032c6e3e5d754be1814dc735864864dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032c6e3e5d754be1814dc735864864dc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36023 } } }
I20260924 23:13:40.858070 28543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [sys.catalog]: This master's current role is: LEADER
I20260924 23:13:40.857940 28557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:13:40.858603 28557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 23:13:40 dist-test-slave-m0z6 krb5kdc[28456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:40.860574 28557 catalog_manager.cc:1347] Generated new cluster ID: f1c9f08f4aba4985847c328c9c70611c
I20260924 23:13:40.860621 28557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:13:40.864857 28541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:40.853147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53486 (local address 127.12.212.62:36023)
0924 23:13:40.853340 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:40.853350 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:40.853385 (+ 35us) server_negotiation.cc:407] Connection header received
0924 23:13:40.854442 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:40.854452 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:40.854516 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:40.854680 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:40.855448 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:40.856543 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:40.857572 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:40.858196 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:40.858289 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:40.860986 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:40.860991 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:40.860991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:40.861004 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:40.863091 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:40.863503 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:40.863506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:40.863507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:40.863643 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:40.863831 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:40.863832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:40.863832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:40.864172 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:40.864336 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:40.864434 (+ 98us) server_negotiation.cc:299] Negotiation successful
0924 23:13:40.864606 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
I20260924 23:13:40.875321 28557 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:13:40.876104 28557 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:13:40.889007 28557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc: Generated new TSK 0
I20260924 23:13:40.889268 28557 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
I20260924 23:13:40.922188 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:36023
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:13:41.011842 28563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:41.012080 28563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:41.012106 28563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:41.013895 28563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:41.014061 28563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:41.014102 28563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:13:41.016108 28563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:36023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:41.016415 28563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:41.016674 28563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:41.021943 28571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:41.021970 28568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:41.022090 28569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:41.022358 28563 server_base.cc:1034] running on GCE node
I20260924 23:13:41.022564 28563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:41.022848 28563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:41.024037 28563 hybrid_clock.cc:648] HybridClock initialized: now 1790291621024019 us; error 30 us; skew 500 ppm
Sep 24 23:13:41 dist-test-slave-m0z6 krb5kdc[28456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291621, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:41.026861 28563 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:13:41.027386 28563 webserver.cc:460] Webserver started at http://127.12.212.1:35621/ using document root <none> and password file <none>
I20260924 23:13:41.027621 28563 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:41.027671 28563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:41.027812 28563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:13:41.028957 28563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e"
format_stamp: "Formatted at 2026-09-24 23:13:41 on dist-test-slave-m0z6"
server_key: "321f561106d05edd2255cbca68661288"
server_key_iv: "f67cfbfe18b7788d80ac5f9edc874459"
server_key_version: "encryptionkey@0"
I20260924 23:13:41.029309 28563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e"
format_stamp: "Formatted at 2026-09-24 23:13:41 on dist-test-slave-m0z6"
server_key: "321f561106d05edd2255cbca68661288"
server_key_iv: "f67cfbfe18b7788d80ac5f9edc874459"
server_key_version: "encryptionkey@0"
I20260924 23:13:41.030716 28563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:13:41.031668 28578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:41.031920 28563 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:41.032025 28563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e"
format_stamp: "Formatted at 2026-09-24 23:13:41 on dist-test-slave-m0z6"
server_key: "321f561106d05edd2255cbca68661288"
server_key_iv: "f67cfbfe18b7788d80ac5f9edc874459"
server_key_version: "encryptionkey@0"
I20260924 23:13:41.032107 28563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:41.064730 28563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:41.067137 28563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:41.067294 28563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:41.067534 28563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:13:41.067844 28563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:13:41.067883 28563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:41.067914 28563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:13:41.067934 28563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:41.073817 28563 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:37997
I20260924 23:13:41.073969 28691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:37997 every 8 connection(s)
I20260924 23:13:41.074227 28563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:13:41.078181 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 28563
I20260924 23:13:41.078321 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:13:41.078539 13136 external_mini_cluster.cc:1428] Setting key 18357c3b2cfa74f7087fe1e0424c38a2
Sep 24 23:13:41 dist-test-slave-m0z6 krb5kdc[28456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291621, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:41.085055 28541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.075127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:53637 (local address 127.12.212.62:36023)
0924 23:13:41.075226 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:41.075228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:41.075411 (+ 183us) server_negotiation.cc:407] Connection header received
0924 23:13:41.075974 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:41.075975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:41.075987 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:41.076047 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:41.077083 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.077595 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.079005 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.079293 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.079363 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:41.081813 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:41.081816 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:41.081816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:41.081829 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:41.083505 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:41.084014 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:41.084017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:41.084017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:41.084077 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:41.084348 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:41.084349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:41.084350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:41.084476 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:41.084536 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:41.084842 (+ 306us) server_negotiation.cc:299] Negotiation successful
0924 23:13:41.084957 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 23:13:41.085232 28694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.075029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:53637)
0924 23:13:41.075253 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:41.075309 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:41.075865 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:41.076129 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:41.076133 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:41.076750 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:41.076972 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:41.076976 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.077699 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.077700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:41.078927 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:41.078930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.079378 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.079379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:41.079509 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:41.080073 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:41.080083 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:41.081679 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:41.083623 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:41.083637 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:41.083640 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:41.083930 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:41.084140 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:41.084141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:41.084141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:41.084285 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:41.084535 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:41.084544 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:41.084709 (+ 165us) client_negotiation.cc:769] Sending connection context
0924 23:13:41.084806 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 23:13:41.084986 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":93,"threads_started":1}
I20260924 23:13:41.086091 28692 heartbeater.cc:347] Connected to a master server at 127.12.212.62:36023
I20260924 23:13:41.086200 28692 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:41.086445 28692 heartbeater.cc:511] Master 127.12.212.62:36023 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.12.212.2@KRBTEST.COM; defaulting to no policy
I20260924 23:13:41.087105 28501 ts_manager.cc:194] Registered new tserver with Master: 3ec283fbe3cd4e0182faa7937a6a8f7e (127.12.212.1:37997)
I20260924 23:13:41.087776 28501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:53637
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
I20260924 23:13:41.106040 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:36023
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:13:41.187444 28697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:41.187687 28697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:41.187716 28697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:41.189059 28697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:41.189126 28697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:41.189155 28697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:13:41.190950 28697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:36023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:41.191234 28697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:41.191478 28697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:41.196416 28705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:41.196447 28702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:41.196449 28703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:41.196559 28697 server_base.cc:1034] running on GCE node
I20260924 23:13:41.196918 28697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:41.197170 28697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:41.198328 28697 hybrid_clock.cc:648] HybridClock initialized: now 1790291621198303 us; error 41 us; skew 500 ppm
Sep 24 23:13:41 dist-test-slave-m0z6 krb5kdc[28456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291621, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:41.201086 28697 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:13:41.201570 28697 webserver.cc:460] Webserver started at http://127.12.212.2:37241/ using document root <none> and password file <none>
I20260924 23:13:41.201781 28697 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:41.201839 28697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:41.201968 28697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:13:41.203120 28697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "3e2fd14b901f4dabb87804e3ed361567"
format_stamp: "Formatted at 2026-09-24 23:13:41 on dist-test-slave-m0z6"
server_key: "b6da2d5bc2f95513db7d4e3135ab58f8"
server_key_iv: "7113fc632277eb922a16e5e5a4f55f4e"
server_key_version: "encryptionkey@0"
I20260924 23:13:41.203481 28697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "3e2fd14b901f4dabb87804e3ed361567"
format_stamp: "Formatted at 2026-09-24 23:13:41 on dist-test-slave-m0z6"
server_key: "b6da2d5bc2f95513db7d4e3135ab58f8"
server_key_iv: "7113fc632277eb922a16e5e5a4f55f4e"
server_key_version: "encryptionkey@0"
I20260924 23:13:41.205113 28697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:13:41.206069 28712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:41.206523 28697 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:41.206631 28697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "3e2fd14b901f4dabb87804e3ed361567"
format_stamp: "Formatted at 2026-09-24 23:13:41 on dist-test-slave-m0z6"
server_key: "b6da2d5bc2f95513db7d4e3135ab58f8"
server_key_iv: "7113fc632277eb922a16e5e5a4f55f4e"
server_key_version: "encryptionkey@0"
I20260924 23:13:41.206759 28697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:41.226284 28697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:41.228788 28697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:41.228919 28697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:41.229112 28697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:13:41.229391 28697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:13:41.229429 28697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:41.229451 28697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:13:41.229462 28697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:41.235129 28697 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:44289
I20260924 23:13:41.235186 28825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:44289 every 8 connection(s)
I20260924 23:13:41.235504 28697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:13:41.241358 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 28697
I20260924 23:13:41.241478 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
Sep 24 23:13:41 dist-test-slave-m0z6 krb5kdc[28456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291621, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:41.241650 13136 external_mini_cluster.cc:1428] Setting key 9cf00771e8d37f39f157641b1f8172d2
I20260924 23:13:41.245038 28541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.236220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:44171 (local address 127.12.212.62:36023)
0924 23:13:41.236278 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:41.236280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:41.236540 (+ 260us) server_negotiation.cc:407] Connection header received
0924 23:13:41.237071 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:41.237072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:41.237084 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:41.237131 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:41.238113 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.238627 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.239654 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.239869 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.239934 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:41.242147 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:41.242149 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:41.242150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:41.242157 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:41.243504 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:41.243965 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:41.243966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:41.243967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:41.244035 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:41.244260 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:41.244261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:41.244261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:41.244405 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:41.244476 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:41.244835 (+ 359us) server_negotiation.cc:299] Negotiation successful
0924 23:13:41.244939 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 23:13:41.245261 28828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.236218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:44171)
0924 23:13:41.236413 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:41.236453 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:41.236967 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:41.237205 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:41.237209 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:41.237803 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:41.238013 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:41.238018 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.238740 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.238741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:41.239566 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:41.239569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.239956 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.239957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:41.240078 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:41.240596 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:41.240605 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:41.242050 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:41.243607 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:41.243620 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:41.243623 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:41.243887 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:41.244094 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:41.244095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:41.244095 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:41.244207 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:41.244511 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:41.244518 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:41.244663 (+ 145us) client_negotiation.cc:769] Sending connection context
0924 23:13:41.244784 (+ 121us) client_negotiation.cc:240] Negotiation successful
0924 23:13:41.245029 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
I20260924 23:13:41.245893 28826 heartbeater.cc:347] Connected to a master server at 127.12.212.62:36023
I20260924 23:13:41.246009 28826 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:41.246222 28826 heartbeater.cc:511] Master 127.12.212.62:36023 requested a full tablet report, sending...
I20260924 23:13:41.246631 28501 ts_manager.cc:194] Registered new tserver with Master: 3e2fd14b901f4dabb87804e3ed361567 (127.12.212.2:44289)
I20260924 23:13:41.247061 28501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:44171
WARNING: no policy specified for kudu/127.12.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
I20260924 23:13:41.267554 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:36023
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:13:41.350167 28831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:13:41.350404 28831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:13:41.350428 28831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:13:41.351719 28831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:13:41.351773 28831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:13:41.351799 28831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:13:41.353314 28831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:36023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:13:41.353531 28831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:13:41.353724 28831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:13:41.358649 28836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:41.358649 28837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:13:41.358644 28839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:13:41.359083 28831 server_base.cc:1034] running on GCE node
I20260924 23:13:41.359293 28831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:13:41.359516 28831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:13:41.360669 28831 hybrid_clock.cc:648] HybridClock initialized: now 1790291621360654 us; error 26 us; skew 500 ppm
Sep 24 23:13:41 dist-test-slave-m0z6 krb5kdc[28456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291621, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:13:41.363294 28831 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:13:41.363791 28831 webserver.cc:460] Webserver started at http://127.12.212.3:44229/ using document root <none> and password file <none>
I20260924 23:13:41.364038 28831 fs_manager.cc:359] Metadata directory not provided
I20260924 23:13:41.364091 28831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:13:41.364217 28831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:13:41.365473 28831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "945c11a977764bfba378cba52cb70039"
format_stamp: "Formatted at 2026-09-24 23:13:41 on dist-test-slave-m0z6"
server_key: "da5099f248bea974ea6205ae8a25d79d"
server_key_iv: "a4f99f80be8e8629036eb99e4c39a022"
server_key_version: "encryptionkey@0"
I20260924 23:13:41.365865 28831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "945c11a977764bfba378cba52cb70039"
format_stamp: "Formatted at 2026-09-24 23:13:41 on dist-test-slave-m0z6"
server_key: "da5099f248bea974ea6205ae8a25d79d"
server_key_iv: "a4f99f80be8e8629036eb99e4c39a022"
server_key_version: "encryptionkey@0"
I20260924 23:13:41.367323 28831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:13:41.368152 28846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:41.368346 28831 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:13:41.368436 28831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "945c11a977764bfba378cba52cb70039"
format_stamp: "Formatted at 2026-09-24 23:13:41 on dist-test-slave-m0z6"
server_key: "da5099f248bea974ea6205ae8a25d79d"
server_key_iv: "a4f99f80be8e8629036eb99e4c39a022"
server_key_version: "encryptionkey@0"
I20260924 23:13:41.368499 28831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:13:41.381701 28831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:13:41.384017 28831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:13:41.384167 28831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:13:41.384418 28831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:13:41.384719 28831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:13:41.384752 28831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:41.384783 28831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:13:41.384799 28831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:13:41.390854 28831 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:44561
I20260924 23:13:41.390911 28959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:44561 every 8 connection(s)
I20260924 23:13:41.391242 28831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:13:41.392532 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 28831
I20260924 23:13:41.392635 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:13:41.392804 13136 external_mini_cluster.cc:1428] Setting key f07ab3d86294835ec0482f84a00ffdb7
Sep 24 23:13:41 dist-test-slave-m0z6 krb5kdc[28456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291621, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:13:41.400667 28541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.391993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:48117 (local address 127.12.212.62:36023)
0924 23:13:41.392088 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:41.392090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:41.392252 (+ 162us) server_negotiation.cc:407] Connection header received
0924 23:13:41.393009 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:41.393010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:41.393025 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:41.393091 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:41.394087 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.394604 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.395690 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.395891 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.395957 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:41.398137 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:41.398145 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:41.398145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:41.398153 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:41.399493 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:41.399865 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:41.399866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:41.399866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:41.399917 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:41.400116 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:41.400117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:41.400117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:41.400226 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:41.400299 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:41.400536 (+ 237us) server_negotiation.cc:299] Negotiation successful
0924 23:13:41.400598 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260924 23:13:41.400862 28962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.391926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:48117)
0924 23:13:41.392121 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:41.392170 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:41.392837 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:41.393224 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:41.393228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:41.393793 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:13:41.394000 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:41.394006 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.394724 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.394725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:41.395591 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:41.395594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.395974 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.395975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:41.396090 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:41.396594 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:13:41.396603 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:13:41.398030 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:13:41.399569 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:41.399576 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:41.399578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:41.399807 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:41.399965 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:13:41.399965 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:13:41.399965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:13:41.400067 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:13:41.400296 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:13:41.400301 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:13:41.400432 (+ 131us) client_negotiation.cc:769] Sending connection context
0924 23:13:41.400514 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 23:13:41.400643 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":85,"threads_started":1}
I20260924 23:13:41.401320 28960 heartbeater.cc:347] Connected to a master server at 127.12.212.62:36023
I20260924 23:13:41.401413 28960 heartbeater.cc:464] Registering TS with master...
I20260924 23:13:41.401595 28960 heartbeater.cc:511] Master 127.12.212.62:36023 requested a full tablet report, sending...
I20260924 23:13:41.402032 28501 ts_manager.cc:194] Registered new tserver with Master: 945c11a977764bfba378cba52cb70039 (127.12.212.3:44561)
I20260924 23:13:41.402372 28501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:48117
I20260924 23:13:41.404433 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 23:13:41.411855 28541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.405681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53490 (local address 127.12.212.62:36023)
0924 23:13:41.405779 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:41.405781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:41.405888 (+ 107us) server_negotiation.cc:407] Connection header received
0924 23:13:41.405958 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:41.405959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:41.405970 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:41.406017 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:13:41.406616 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.407167 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.407875 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.408085 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.408139 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:41.409278 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:13:41.409280 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:13:41.409280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:13:41.409288 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:13:41.410572 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:41.410972 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:41.410974 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:41.410974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:41.411040 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:13:41.411210 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:13:41.411211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:13:41.411211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:13:41.411424 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:13:41.411525 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:41.411638 (+ 113us) server_negotiation.cc:299] Negotiation successful
0924 23:13:41.411723 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260924 23:13:41.413614 28501 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:13:41.415395 28501 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:13:41.422587 28974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.418141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46392 (local address 127.12.212.3:44561)
0924 23:13:41.418351 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:41.418356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:41.418629 (+ 273us) server_negotiation.cc:407] Connection header received
0924 23:13:41.418725 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:41.418727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:41.418744 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:41.418813 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:13:41.419091 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.420248 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.421458 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.422186 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.422256 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:41.422314 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:41.422455 (+ 141us) server_negotiation.cc:299] Negotiation successful
0924 23:13:41.422498 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":89,"threads_started":1}
I20260924 23:13:41.422585 28975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.417981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:44561 (local address 127.0.0.1:46392)
0924 23:13:41.418562 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:41.418573 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:41.418631 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:41.418886 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:41.418886 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:41.418888 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:13:41.418996 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:41.418998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.420371 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.420372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:41.421381 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:41.421384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.422277 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.422278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:41.422348 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:41.422360 (+ 12us) client_negotiation.cc:769] Sending connection context
0924 23:13:41.422427 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 23:13:41.422490 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"mutex_wait_us":9,"thread_start_us":49,"threads_started":1}
I20260924 23:13:41.422753 28972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.417978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:37997 (local address 127.0.0.1:34492)
0924 23:13:41.418145 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:41.418195 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:41.418265 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:41.418490 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:41.418494 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:41.418497 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:13:41.418585 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:41.418587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.419691 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.419692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:41.420638 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:41.420641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.422489 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.422491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:41.422563 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:41.422569 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 23:13:41.422616 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 23:13:41.422661 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":105,"threads_started":1}
I20260924 23:13:41.422966 28973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.418090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34492 (local address 127.12.212.1:37997)
0924 23:13:41.418288 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:41.418294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:41.418307 (+ 13us) server_negotiation.cc:407] Connection header received
0924 23:13:41.418354 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:41.418356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:41.418373 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:41.418463 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:13:41.418681 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.419588 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.420751 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.421526 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.421604 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:41.421664 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:41.422852 (+ 1188us) server_negotiation.cc:299] Negotiation successful
0924 23:13:41.422901 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":78,"threads_started":1}
I20260924 23:13:41.422969 28976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.418376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59628 (local address 127.12.212.2:44289)
0924 23:13:41.418564 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:41.418568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:41.418849 (+ 281us) server_negotiation.cc:407] Connection header received
0924 23:13:41.418893 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:41.418896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:41.418912 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:41.418980 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:13:41.419193 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.420284 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.421613 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.422337 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.422395 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:41.422451 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:41.422633 (+ 182us) server_negotiation.cc:299] Negotiation successful
0924 23:13:41.422894 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":100,"threads_started":1}
I20260924 23:13:41.423575 28894 tablet_service.cc:1467] Processing CreateTablet for tablet b99e697aa8bd42c18c0c93c44f813da1 (DEFAULT_TABLE table=test-table [id=88dbf8595f2849a9a7e6788b75074b65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:13:41.423607 28626 tablet_service.cc:1467] Processing CreateTablet for tablet b99e697aa8bd42c18c0c93c44f813da1 (DEFAULT_TABLE table=test-table [id=88dbf8595f2849a9a7e6788b75074b65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:13:41.423890 28626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99e697aa8bd42c18c0c93c44f813da1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:41.423898 28894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99e697aa8bd42c18c0c93c44f813da1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:41.426311 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.418182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:44289 (local address 127.0.0.1:59628)
0924 23:13:41.418745 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:41.418753 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:41.418799 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:41.419059 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:41.419060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:41.419063 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:13:41.419110 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:41.419112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.420390 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.420391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:41.421533 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:41.421536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.422441 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.422442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:41.422495 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:41.422499 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:13:41.422547 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 23:13:41.422581 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":56,"thread_start_us":56,"threads_started":1}
I20260924 23:13:41.426791 28978 tablet_bootstrap.cc:492] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039: Bootstrap starting.
I20260924 23:13:41.426792 28977 tablet_bootstrap.cc:492] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Bootstrap starting.
I20260924 23:13:41.427134 28760 tablet_service.cc:1467] Processing CreateTablet for tablet b99e697aa8bd42c18c0c93c44f813da1 (DEFAULT_TABLE table=test-table [id=88dbf8595f2849a9a7e6788b75074b65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:13:41.427381 28978 tablet_bootstrap.cc:654] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039: Neither blocks nor log segments found. Creating new log.
I20260924 23:13:41.427414 28977 tablet_bootstrap.cc:654] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Neither blocks nor log segments found. Creating new log.
I20260924 23:13:41.427546 28760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99e697aa8bd42c18c0c93c44f813da1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:13:41.427943 28978 log.cc:824] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:41.427973 28977 log.cc:824] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:41.428839 28977 tablet_bootstrap.cc:492] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e: No bootstrap required, opened a new log
I20260924 23:13:41.428910 28977 ts_tablet_manager.cc:1397] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 23:13:41.428972 28978 tablet_bootstrap.cc:492] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039: No bootstrap required, opened a new log
I20260924 23:13:41.429044 28978 ts_tablet_manager.cc:1397] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 23:13:41.430521 28978 raft_consensus.cc:348] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } }
I20260924 23:13:41.430673 28978 raft_consensus.cc:374] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:13:41.430735 28978 raft_consensus.cc:729] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945c11a977764bfba378cba52cb70039, State: Initialized, Role: FOLLOWER
I20260924 23:13:41.430886 28978 consensus_queue.cc:260] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } }
I20260924 23:13:41.430903 28977 raft_consensus.cc:348] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } }
I20260924 23:13:41.431082 28977 raft_consensus.cc:374] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:13:41.431113 28977 raft_consensus.cc:729] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ec283fbe3cd4e0182faa7937a6a8f7e, State: Initialized, Role: FOLLOWER
I20260924 23:13:41.431193 28978 ts_tablet_manager.cc:1428] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 23:13:41.431232 28977 consensus_queue.cc:260] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } }
I20260924 23:13:41.431366 28960 heartbeater.cc:503] Master 127.12.212.62:36023 was elected leader, sending a full tablet report...
I20260924 23:13:41.431527 28977 ts_tablet_manager.cc:1428] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260924 23:13:41.431541 28692 heartbeater.cc:503] Master 127.12.212.62:36023 was elected leader, sending a full tablet report...
I20260924 23:13:41.431723 28981 tablet_bootstrap.cc:492] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567: Bootstrap starting.
I20260924 23:13:41.432368 28981 tablet_bootstrap.cc:654] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567: Neither blocks nor log segments found. Creating new log.
I20260924 23:13:41.432875 28981 log.cc:824] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567: Log is configured to *not* fsync() on all Append() calls
I20260924 23:13:41.433554 28981 tablet_bootstrap.cc:492] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567: No bootstrap required, opened a new log
I20260924 23:13:41.433610 28981 ts_tablet_manager.cc:1397] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:13:41.435096 28981 raft_consensus.cc:348] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } }
I20260924 23:13:41.435243 28981 raft_consensus.cc:374] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:13:41.435263 28981 raft_consensus.cc:729] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e2fd14b901f4dabb87804e3ed361567, State: Initialized, Role: FOLLOWER
I20260924 23:13:41.435372 28981 consensus_queue.cc:260] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } }
I20260924 23:13:41.435663 28981 ts_tablet_manager.cc:1428] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:13:41.435700 28826 heartbeater.cc:503] Master 127.12.212.62:36023 was elected leader, sending a full tablet report...
I20260924 23:13:41.472050 28982 raft_consensus.cc:482] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:13:41.472252 28982 raft_consensus.cc:504] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } }
I20260924 23:13:41.472645 28982 leader_election.cc:288] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ec283fbe3cd4e0182faa7937a6a8f7e (127.12.212.1:37997), 3e2fd14b901f4dabb87804e3ed361567 (127.12.212.2:44289)
I20260924 23:13:41.476660 28973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.473003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:49963 (local address 127.12.212.1:37997)
0924 23:13:41.473053 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:41.473056 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:41.473067 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:13:41.473191 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:41.473192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:41.473203 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:41.473259 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:13:41.473440 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.474173 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.475684 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.476286 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.476374 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:41.476394 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:41.476544 (+ 150us) server_negotiation.cc:299] Negotiation successful
0924 23:13:41.476585 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 23:13:41.476660 28976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.473097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:34049 (local address 127.12.212.2:44289)
0924 23:13:41.473144 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:41.473147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:41.473261 (+ 114us) server_negotiation.cc:407] Connection header received
0924 23:13:41.473339 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:41.473340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:41.473350 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:41.473390 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:13:41.473524 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.474206 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.475685 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.476319 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.476388 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:41.476397 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:41.476545 (+ 148us) server_negotiation.cc:299] Negotiation successful
0924 23:13:41.476584 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 23:13:41.476679 28962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.472917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:37997 (local address 127.12.212.3:49963)
0924 23:13:41.472993 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:41.473006 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:41.473072 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:41.473303 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:41.473303 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:41.473305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:13:41.473374 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:41.473376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.474255 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.474256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:41.475566 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:41.475570 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.476396 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.476397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:41.476475 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:41.476477 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 23:13:41.476534 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 23:13:41.476584 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":36}
I20260924 23:13:41.476687 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.473023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:44289 (local address 127.12.212.3:34049)
0924 23:13:41.473194 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:41.473202 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:13:41.473256 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:13:41.473411 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:13:41.473411 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:13:41.473413 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:13:41.473467 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:41.473468 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.474267 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.474268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:41.475566 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:13:41.475570 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.476390 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.476391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:13:41.476472 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:41.476477 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 23:13:41.476534 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 23:13:41.476584 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":8,"thread_start_us":121,"threads_started":1}
I20260924 23:13:41.477142 28646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99e697aa8bd42c18c0c93c44f813da1" candidate_uuid: "945c11a977764bfba378cba52cb70039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" is_pre_election: true
I20260924 23:13:41.477321 28646 raft_consensus.cc:2454] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 945c11a977764bfba378cba52cb70039 in term 0.
I20260924 23:13:41.477278 28780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99e697aa8bd42c18c0c93c44f813da1" candidate_uuid: "945c11a977764bfba378cba52cb70039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e2fd14b901f4dabb87804e3ed361567" is_pre_election: true
I20260924 23:13:41.477424 28780 raft_consensus.cc:2454] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 945c11a977764bfba378cba52cb70039 in term 0.
I20260924 23:13:41.477581 28847 leader_election.cc:302] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ec283fbe3cd4e0182faa7937a6a8f7e, 945c11a977764bfba378cba52cb70039; no voters:
I20260924 23:13:41.477771 28982 raft_consensus.cc:2781] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:13:41.477836 28982 raft_consensus.cc:482] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:13:41.477851 28982 raft_consensus.cc:3037] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:13:41.478677 28982 raft_consensus.cc:504] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } }
I20260924 23:13:41.478823 28982 leader_election.cc:288] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [CANDIDATE]: Term 1 election: Requested vote from peers 3ec283fbe3cd4e0182faa7937a6a8f7e (127.12.212.1:37997), 3e2fd14b901f4dabb87804e3ed361567 (127.12.212.2:44289)
I20260924 23:13:41.478978 28646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99e697aa8bd42c18c0c93c44f813da1" candidate_uuid: "945c11a977764bfba378cba52cb70039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e"
I20260924 23:13:41.478997 28780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99e697aa8bd42c18c0c93c44f813da1" candidate_uuid: "945c11a977764bfba378cba52cb70039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e2fd14b901f4dabb87804e3ed361567"
I20260924 23:13:41.479063 28780 raft_consensus.cc:3037] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:13:41.479064 28646 raft_consensus.cc:3037] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:13:41.479791 28780 raft_consensus.cc:2454] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 945c11a977764bfba378cba52cb70039 in term 1.
I20260924 23:13:41.479826 28646 raft_consensus.cc:2454] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 945c11a977764bfba378cba52cb70039 in term 1.
I20260924 23:13:41.479959 28847 leader_election.cc:302] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ec283fbe3cd4e0182faa7937a6a8f7e, 945c11a977764bfba378cba52cb70039; no voters:
I20260924 23:13:41.480046 28982 raft_consensus.cc:2781] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:13:41.480212 28982 raft_consensus.cc:686] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [term 1 LEADER]: Becoming Leader. State: Replica: 945c11a977764bfba378cba52cb70039, State: Running, Role: LEADER
I20260924 23:13:41.480305 28982 consensus_queue.cc:237] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } }
I20260924 23:13:41.481132 28500 catalog_manager.cc:5495] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 reported cstate change: term changed from 0 to 1, leader changed from <none> to 945c11a977764bfba378cba52cb70039 (127.12.212.3). New cstate: current_term: 1 leader_uuid: "945c11a977764bfba378cba52cb70039" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } health_report { overall_health: HEALTHY } } }
W20260924 23:13:41.486132 28827 tablet.cc:2369] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:13:41.498449 28974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.495911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46400 (local address 127.12.212.3:44561)
0924 23:13:41.495962 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 23:13:41.495964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:13:41.495972 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:13:41.496008 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:13:41.496008 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:13:41.496026 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:13:41.496089 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:13:41.496250 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.496725 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.497499 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:13:41.497695 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:13:41.497751 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:13:41.497970 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:13:41.498091 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:13:41.498161 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 23:13:41.498300 (+ 139us) server_negotiation.cc:299] Negotiation successful
0924 23:13:41.498331 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 23:13:41.500479 28780 raft_consensus.cc:1261] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567 [term 1 FOLLOWER]: Refusing update from remote peer 945c11a977764bfba378cba52cb70039: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:13:41.500479 28646 raft_consensus.cc:1261] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Refusing update from remote peer 945c11a977764bfba378cba52cb70039: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:13:41.500893 28987 consensus_queue.cc:1035] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:13:41.501051 28982 consensus_queue.cc:1035] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:13:41.506249 28500 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:13:41.506392 28500 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:13:41.507179 28500 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc: Sending DeleteTablet for 3 replicas of tablet b99e697aa8bd42c18c0c93c44f813da1
I20260924 23:13:41.507612 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 28563
I20260924 23:13:41.507580 28894 tablet_service.cc:1514] Processing DeleteTablet for tablet b99e697aa8bd42c18c0c93c44f813da1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:13:41 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:46392
I20260924 23:13:41.507813 28760 tablet_service.cc:1514] Processing DeleteTablet for tablet b99e697aa8bd42c18c0c93c44f813da1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:13:41 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:59628
I20260924 23:13:41.508078 29000 tablet_replica.cc:330] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567: stopping tablet replica
I20260924 23:13:41.508198 29000 raft_consensus.cc:2229] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:13:41.508335 29000 raft_consensus.cc:2258] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:13:41.508843 29000 ts_tablet_manager.cc:1905] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:13:41.510635 29000 ts_tablet_manager.cc:1918] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:13:41.510751 29000 log.cc:1196] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/b99e697aa8bd42c18c0c93c44f813da1
I20260924 23:13:41.511026 29000 ts_tablet_manager.cc:1939] T b99e697aa8bd42c18c0c93c44f813da1 P 3e2fd14b901f4dabb87804e3ed361567: Deleting consensus metadata
I20260924 23:13:41.511603 28485 catalog_manager.cc:4841] TS 3e2fd14b901f4dabb87804e3ed361567 (127.12.212.2:44289): tablet b99e697aa8bd42c18c0c93c44f813da1 (table test-table [id=88dbf8595f2849a9a7e6788b75074b65]) successfully deleted
W20260924 23:13:41.513424 28485 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.212.1:37997 (error 108)
I20260924 23:13:41.513537 28999 tablet_replica.cc:330] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039: stopping tablet replica
I20260924 23:13:41.513659 28999 raft_consensus.cc:2229] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:13:41.513785 28999 raft_consensus.cc:2258] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:13:41.514002 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 28697
I20260924 23:13:41.514048 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.513917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:37997 (local address 127.0.0.1:34496)
0924 23:13:41.513972 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:41.514013 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.1:37997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260924 23:13:41.514248 28485 catalog_manager.cc:4584] TS 3ec283fbe3cd4e0182faa7937a6a8f7e (127.12.212.1:37997): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b99e697aa8bd42c18c0c93c44f813da1: Network error: Client connection negotiation failed: client connection to 127.12.212.1:37997: connect: Connection refused (error 111)
I20260924 23:13:41.516412 28999 ts_tablet_manager.cc:1905] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:13:41.518033 28999 ts_tablet_manager.cc:1918] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:13:41.518111 28999 log.cc:1196] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/b99e697aa8bd42c18c0c93c44f813da1
I20260924 23:13:41.518090 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 28831
I20260924 23:13:41.522192 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 28469
I20260924 23:13:41.522349 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:13:41.522211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:44561 (local address 127.0.0.1:46406)
0924 23:13:41.522279 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0924 23:13:41.522313 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.3:44561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260924 23:14:01.527101 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:36023
--webserver_interface=127.12.212.62
--webserver_port=35835
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:36023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:01.610503 29002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:01.610775 29002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:01.610808 29002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:01.612113 29002 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 23:14:01.612162 29002 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 23:14:01.612282 29002 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 23:14:01.612309 29002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:14:01.612335 29002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:01.612362 29002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:01.612380 29002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:14:01.612404 29002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:14:01.614228 29002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:36023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:36023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=35835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:01.614531 29002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:01.614795 29002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:01.623574 29009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:01.623565 29008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:01.623611 29002 server_base.cc:1034] running on GCE node
W20260924 23:14:01.623572 29011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:01.624177 29002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:01.624517 29002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:01.625684 29002 hybrid_clock.cc:648] HybridClock initialized: now 1790291641625671 us; error 30 us; skew 500 ppm
Sep 24 23:14:01 dist-test-slave-m0z6 krb5kdc[28456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291641, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:01.628520 29002 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:14:01.629000 29002 webserver.cc:460] Webserver started at http://127.12.212.62:35835/ using document root <none> and password file <none>
I20260924 23:14:01.629257 29002 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:01.629317 29002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:01.630779 29002 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:14:01.631464 29018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:01.631676 29002 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:01.631776 29002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "032c6e3e5d754be1814dc735864864dc"
format_stamp: "Formatted at 2026-09-24 23:13:40 on dist-test-slave-m0z6"
server_key: "6f4b5b4eb1f2b12c8ce70ca717554283"
server_key_iv: "8db77c7fb2ce2aedbc222f834e64f712"
server_key_version: "encryptionkey@0"
I20260924 23:14:01.632202 29002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:01.645887 29002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:01.648501 29002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:01.648633 29002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:01.652972 29002 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:36023
I20260924 23:14:01.653024 29070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:36023 every 8 connection(s)
I20260924 23:14:01.653431 29002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:14:01.655082 29071 sys_catalog.cc:263] Verifying existing consensus state
I20260924 23:14:01.655814 29071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc: Bootstrap starting.
I20260924 23:14:01.662750 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 29002
I20260924 23:14:01.663133 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:37997
--local_ip_for_outbound_sockets=127.12.212.1
--tserver_master_addrs=127.12.212.62:36023
--webserver_port=35621
--webserver_interface=127.12.212.1
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 23:14:01.664600 29071 log.cc:824] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:01.667014 29071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 23:14:01.667403 29071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc: Bootstrap complete.
I20260924 23:14:01.669736 29071 raft_consensus.cc:348] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032c6e3e5d754be1814dc735864864dc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36023 } }
I20260924 23:14:01.669996 29071 raft_consensus.cc:729] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 032c6e3e5d754be1814dc735864864dc, State: Initialized, Role: FOLLOWER
I20260924 23:14:01.670145 29071 consensus_queue.cc:260] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032c6e3e5d754be1814dc735864864dc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36023 } }
I20260924 23:14:01.670212 29071 raft_consensus.cc:388] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:14:01.670279 29071 raft_consensus.cc:482] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:14:01.670333 29071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 1 FOLLOWER]: Advancing to term 2
I20260924 23:14:01.671494 29071 raft_consensus.cc:504] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032c6e3e5d754be1814dc735864864dc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36023 } }
I20260924 23:14:01.671592 29071 leader_election.cc:302] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 032c6e3e5d754be1814dc735864864dc; no voters:
I20260924 23:14:01.671773 29071 leader_election.cc:288] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 23:14:01.671830 29075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 2 FOLLOWER]: Leader election won for term 2
I20260924 23:14:01.671952 29075 raft_consensus.cc:686] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 2 LEADER]: Becoming Leader. State: Replica: 032c6e3e5d754be1814dc735864864dc, State: Running, Role: LEADER
I20260924 23:14:01.672060 29071 sys_catalog.cc:564] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:14:01.672039 29075 consensus_queue.cc:237] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032c6e3e5d754be1814dc735864864dc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36023 } }
I20260924 23:14:01.672387 29075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 032c6e3e5d754be1814dc735864864dc. Latest consensus state: current_term: 2 leader_uuid: "032c6e3e5d754be1814dc735864864dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032c6e3e5d754be1814dc735864864dc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36023 } } }
I20260924 23:14:01.672482 29075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:01.672711 29076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "032c6e3e5d754be1814dc735864864dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032c6e3e5d754be1814dc735864864dc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36023 } } }
I20260924 23:14:01.672813 29076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:01.672955 29081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:14:01.673767 29081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:14:01.673875 29081 catalog_manager.cc:1259] Loaded cluster ID: f1c9f08f4aba4985847c328c9c70611c
I20260924 23:14:01.673903 29081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:14:01.675153 29081 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:14:01.675352 29081 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc: Loaded TSK: 0
I20260924 23:14:01.691558 29081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc: Generated new TSK 1
I20260924 23:14:01.691694 29081 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 23:14:01.754880 29073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:01.755110 29073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:01.755136 29073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:01.756381 29073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:01.756434 29073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:01.756464 29073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:14:01.758024 29073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:37997
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=35621
--tserver_master_addrs=127.12.212.62:36023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:01.758291 29073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:01.758518 29073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:01.763933 29098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:01.763955 29099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:01.763885 29101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:01.764245 29073 server_base.cc:1034] running on GCE node
I20260924 23:14:01.764477 29073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:01.764770 29073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:01.765939 29073 hybrid_clock.cc:648] HybridClock initialized: now 1790291641765924 us; error 29 us; skew 500 ppm
Sep 24 23:14:01 dist-test-slave-m0z6 krb5kdc[28456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291641, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:01.768700 29073 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:14:01.769342 29073 webserver.cc:460] Webserver started at http://127.12.212.1:35621/ using document root <none> and password file <none>
I20260924 23:14:01.769603 29073 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:01.769676 29073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:01.771436 29073 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:14:01.772284 29108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:01.772456 29073 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:01.772552 29073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e"
format_stamp: "Formatted at 2026-09-24 23:13:41 on dist-test-slave-m0z6"
server_key: "321f561106d05edd2255cbca68661288"
server_key_iv: "f67cfbfe18b7788d80ac5f9edc874459"
server_key_version: "encryptionkey@0"
I20260924 23:14:01.773021 29073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:01.805039 29073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:01.807685 29073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:01.807829 29073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:01.808085 29073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:01.808601 29115 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 23:14:01.810242 29073 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 23:14:01.810316 29073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 23:14:01.810348 29073 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 23:14:01.811187 29073 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 23:14:01.811231 29073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:01.811362 29115 tablet_bootstrap.cc:492] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Bootstrap starting.
I20260924 23:14:01.817981 29073 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:37997
I20260924 23:14:01.818071 29222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:37997 every 8 connection(s)
I20260924 23:14:01.818413 29073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:14:01.819216 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 29073
I20260924 23:14:01.819567 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:44289
--local_ip_for_outbound_sockets=127.12.212.2
--tserver_master_addrs=127.12.212.62:36023
--webserver_port=37241
--webserver_interface=127.12.212.2
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 23:14:01.821369 29115 log.cc:824] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:01.823791 29115 tablet_bootstrap.cc:492] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 23:14:01.824070 29115 tablet_bootstrap.cc:492] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Bootstrap complete.
I20260924 23:14:01.824827 29115 ts_tablet_manager.cc:1397] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.011s
I20260924 23:14:01.826130 29115 raft_consensus.cc:348] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } }
I20260924 23:14:01.826481 29115 raft_consensus.cc:729] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ec283fbe3cd4e0182faa7937a6a8f7e, State: Initialized, Role: FOLLOWER
I20260924 23:14:01.826613 29115 consensus_queue.cc:260] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } }
I20260924 23:14:01.826893 29115 ts_tablet_manager.cc:1428] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
Sep 24 23:14:01 dist-test-slave-m0z6 krb5kdc[28456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291641, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:14:01.831730 29225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:01.819130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:57039)
0924 23:14:01.819729 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:01.819788 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:01.820288 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:01.820706 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:01.820710 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:01.821067 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:01.821348 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:01.821355 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:01.822914 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:01.822915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:01.824143 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:01.824147 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:01.825044 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:01.825046 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:01.825187 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:01.825805 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:14:01.825816 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:01.827569 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:01.829939 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:01.829946 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:01.829948 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:01.830185 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:01.830527 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:01.830528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:01.830528 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:01.830637 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:01.831149 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:01.831156 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:01.831336 (+ 180us) client_negotiation.cc:769] Sending connection context
0924 23:14:01.831433 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 23:14:01.831570 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":89,"threads_started":1}
I20260924 23:14:01.831838 29226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:01.819250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:57039 (local address 127.12.212.62:36023)
0924 23:14:01.819622 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:01.819634 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:01.819892 (+ 258us) server_negotiation.cc:407] Connection header received
0924 23:14:01.820505 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:01.820511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:01.820555 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:01.820671 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:01.821489 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:01.822787 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:01.824247 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:01.824898 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:01.825044 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:01.827692 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:01.827697 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:01.827697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:01.827712 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:01.829852 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:01.830318 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:01.830322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:01.830323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:01.830458 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:01.830719 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:01.830720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:01.830720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:01.831061 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:01.831150 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:01.831498 (+ 348us) server_negotiation.cc:299] Negotiation successful
0924 23:14:01.831637 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":92,"threads_started":1}
I20260924 23:14:01.832517 29223 heartbeater.cc:347] Connected to a master server at 127.12.212.62:36023
I20260924 23:14:01.832626 29223 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:01.832810 29223 heartbeater.cc:511] Master 127.12.212.62:36023 requested a full tablet report, sending...
I20260924 23:14:01.833575 29035 ts_manager.cc:194] Registered new tserver with Master: 3ec283fbe3cd4e0182faa7937a6a8f7e (127.12.212.1:37997)
I20260924 23:14:01.834790 29035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:57039
I20260924 23:14:01.835863 29223 heartbeater.cc:503] Master 127.12.212.62:36023 was elected leader, sending a full tablet report...
Sep 24 23:14:01 dist-test-slave-m0z6 krb5kdc[28456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291641, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for kudu/127.12.212.1@KRBTEST.COM
I20260924 23:14:01.843703 29230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:01.834191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:37997 (local address 127.0.0.1:46714)
0924 23:14:01.834500 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:01.834537 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:01.834622 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:01.834985 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:01.834991 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:01.835330 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:01.835405 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:01.835408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:01.836455 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:01.836457 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:01.837357 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:01.837360 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:01.837880 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:01.837881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:01.837971 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:01.838512 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:14:01.838517 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:01.840056 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:01.841796 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:01.841807 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:01.841809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:01.842183 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:01.842628 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:01.842629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:01.842630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:01.842758 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:01.843160 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:01.843169 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:01.843307 (+ 138us) client_negotiation.cc:769] Sending connection context
0924 23:14:01.843411 (+ 104us) client_negotiation.cc:240] Negotiation successful
0924 23:14:01.843582 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":104,"threads_started":1}
I20260924 23:14:01.843737 29231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:01.834254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46714 (local address 127.12.212.1:37997)
0924 23:14:01.834766 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:01.834771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:01.834780 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:14:01.834816 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:01.834818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:01.834844 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:01.834912 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:01.835512 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:01.836390 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:01.837490 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:01.837803 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:01.837869 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:01.840153 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:01.840164 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:01.840165 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:01.840172 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:01.841700 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:01.842295 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:01.842298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:01.842299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:01.842473 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:01.842813 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:01.842814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:01.842815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:01.843064 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:01.843167 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:01.843496 (+ 329us) server_negotiation.cc:299] Negotiation successful
0924 23:14:01.843599 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":87,"threads_started":1}
I20260924 23:14:01.844126 29157 tablet_service.cc:1514] Processing DeleteTablet for tablet b99e697aa8bd42c18c0c93c44f813da1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:13:41 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:46714
I20260924 23:14:01.844642 29232 tablet_replica.cc:330] stopping tablet replica
I20260924 23:14:01.844761 29232 raft_consensus.cc:2229] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:14:01.844833 29232 raft_consensus.cc:2258] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:14:01.845132 29232 ts_tablet_manager.cc:1905] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:14:01.847559 29232 ts_tablet_manager.cc:1918] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:14:01.847640 29232 log.cc:1196] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/wal/wals/b99e697aa8bd42c18c0c93c44f813da1
I20260924 23:14:01.847837 29232 ts_tablet_manager.cc:1939] T b99e697aa8bd42c18c0c93c44f813da1 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Deleting consensus metadata
I20260924 23:14:01.848793 29019 catalog_manager.cc:4841] TS 3ec283fbe3cd4e0182faa7937a6a8f7e (127.12.212.1:37997): tablet b99e697aa8bd42c18c0c93c44f813da1 (table test-table [id=88dbf8595f2849a9a7e6788b75074b65]) successfully deleted
W20260924 23:14:01.911615 29227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:01.911859 29227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:01.911895 29227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:01.913192 29227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:01.913254 29227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:01.913295 29227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:14:01.915427 29227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:44289
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=37241
--tserver_master_addrs=127.12.212.62:36023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:01.915715 29227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:01.915980 29227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:01.921229 29238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:01.921298 29237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:01.921469 29227 server_base.cc:1034] running on GCE node
W20260924 23:14:01.921229 29240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:01.921751 29227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:01.922025 29227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:01.923180 29227 hybrid_clock.cc:648] HybridClock initialized: now 1790291641923168 us; error 28 us; skew 500 ppm
Sep 24 23:14:01 dist-test-slave-m0z6 krb5kdc[28456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291641, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:01.926024 29227 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:14:01.926590 29227 webserver.cc:460] Webserver started at http://127.12.212.2:37241/ using document root <none> and password file <none>
I20260924 23:14:01.926851 29227 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:01.926923 29227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:01.928421 29227 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:01.929183 29247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:01.929385 29227 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:01.929484 29227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "3e2fd14b901f4dabb87804e3ed361567"
format_stamp: "Formatted at 2026-09-24 23:13:41 on dist-test-slave-m0z6"
server_key: "b6da2d5bc2f95513db7d4e3135ab58f8"
server_key_iv: "7113fc632277eb922a16e5e5a4f55f4e"
server_key_version: "encryptionkey@0"
I20260924 23:14:01.929857 29227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:01.943343 29227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:01.945631 29227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:01.945781 29227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:01.946022 29227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:01.946349 29227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:01.946385 29227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:01.946415 29227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:01.946439 29227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:01.952147 29227 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:44289
I20260924 23:14:01.952198 29360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:44289 every 8 connection(s)
I20260924 23:14:01.952540 29227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:14:01.956670 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 29227
I20260924 23:14:01.956964 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:44561
--local_ip_for_outbound_sockets=127.12.212.3
--tserver_master_addrs=127.12.212.62:36023
--webserver_port=44229
--webserver_interface=127.12.212.3
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 23:14:01 dist-test-slave-m0z6 krb5kdc[28456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291641, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:14:01.963168 29226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:01.953363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:51523 (local address 127.12.212.62:36023)
0924 23:14:01.953463 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:01.953466 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:01.953637 (+ 171us) server_negotiation.cc:407] Connection header received
0924 23:14:01.954159 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:01.954161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:01.954176 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:01.954229 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:01.955269 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:01.955781 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:01.956819 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:01.957052 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:01.957119 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:01.959764 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:01.959767 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:01.959767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:01.959775 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:01.961477 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:01.961955 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:01.961956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:01.961957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:01.962024 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:01.962335 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:01.962336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:01.962337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:01.962517 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:01.962589 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:01.962958 (+ 369us) server_negotiation.cc:299] Negotiation successful
0924 23:14:01.963065 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 23:14:01.963352 29363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:01.953256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:51523)
0924 23:14:01.953487 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:01.953538 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:01.954054 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:01.954305 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:01.954309 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:01.954953 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:01.955162 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:01.955168 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:01.955872 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:01.955874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:01.956679 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:01.956681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:01.957207 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:01.957209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:01.957385 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:01.957949 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:14:01.957960 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:01.959633 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:01.961593 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:01.961602 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:01.961605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:01.961875 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:01.962121 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:01.962122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:01.962123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:01.962266 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:01.962585 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:01.962591 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:01.962824 (+ 233us) client_negotiation.cc:769] Sending connection context
0924 23:14:01.962921 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 23:14:01.963070 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":99,"threads_started":1}
I20260924 23:14:01.963831 29361 heartbeater.cc:347] Connected to a master server at 127.12.212.62:36023
I20260924 23:14:01.963922 29361 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:01.964112 29361 heartbeater.cc:511] Master 127.12.212.62:36023 requested a full tablet report, sending...
I20260924 23:14:01.964509 29035 ts_manager.cc:194] Registered new tserver with Master: 3e2fd14b901f4dabb87804e3ed361567 (127.12.212.2:44289)
I20260924 23:14:01.964884 29035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:51523
W20260924 23:14:02.041693 29364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:02.041926 29364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:02.041961 29364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:02.043371 29364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:02.043426 29364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:02.043465 29364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:14:02.045138 29364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:44561
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=44229
--tserver_master_addrs=127.12.212.62:36023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:02.045440 29364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:02.045676 29364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:02.050901 29370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:02.050889 29372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:02.050894 29369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:02.050930 29364 server_base.cc:1034] running on GCE node
I20260924 23:14:02.051394 29364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:02.051654 29364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:02.052819 29364 hybrid_clock.cc:648] HybridClock initialized: now 1790291642052790 us; error 38 us; skew 500 ppm
Sep 24 23:14:02 dist-test-slave-m0z6 krb5kdc[28456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291642, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:02.055465 29364 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:14:02.055956 29364 webserver.cc:460] Webserver started at http://127.12.212.3:44229/ using document root <none> and password file <none>
I20260924 23:14:02.056166 29364 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:02.056288 29364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:02.057682 29364 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:02.058511 29379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:02.058800 29364 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:02.058912 29364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "945c11a977764bfba378cba52cb70039"
format_stamp: "Formatted at 2026-09-24 23:13:41 on dist-test-slave-m0z6"
server_key: "da5099f248bea974ea6205ae8a25d79d"
server_key_iv: "a4f99f80be8e8629036eb99e4c39a022"
server_key_version: "encryptionkey@0"
I20260924 23:14:02.059388 29364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:02.091825 29364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:02.094324 29364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:02.094487 29364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:02.094743 29364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:02.095165 29386 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260924 23:14:02.096686 29386 ts_tablet_manager.cc:1852] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260924 23:14:02.096976 29386 ts_tablet_manager.cc:1905] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:14:02.099193 29386 ts_tablet_manager.cc:1918] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:14:02.099331 29386 log.cc:1196] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/b99e697aa8bd42c18c0c93c44f813da1
I20260924 23:14:02.099589 29386 ts_tablet_manager.cc:1939] T b99e697aa8bd42c18c0c93c44f813da1 P 945c11a977764bfba378cba52cb70039: Deleting consensus metadata
I20260924 23:14:02.099795 29364 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260924 23:14:02.099848 29364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260924 23:14:02.099893 29364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:02.099918 29364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:02.105739 29364 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:44561
I20260924 23:14:02.105803 29493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:44561 every 8 connection(s)
I20260924 23:14:02.106137 29364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
Sep 24 23:14:02 dist-test-slave-m0z6 krb5kdc[28456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291642, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:14:02.113147 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 29364
I20260924 23:14:02.115123 29226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:02.106930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:39129 (local address 127.12.212.62:36023)
0924 23:14:02.106992 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:02.106994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:02.107194 (+ 200us) server_negotiation.cc:407] Connection header received
0924 23:14:02.107712 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:02.107713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:02.107723 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:02.107772 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:02.108362 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.108861 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.109858 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.110055 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.110115 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:02.112257 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:02.112258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:02.112259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:02.112266 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:02.113743 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:02.114141 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:02.114142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:02.114142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:02.114203 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:02.114425 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:02.114426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:02.114427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:02.114542 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:02.114596 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:02.114931 (+ 335us) server_negotiation.cc:299] Negotiation successful
0924 23:14:02.115031 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 23:14:02.115290 29496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:02.106915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:39129)
0924 23:14:02.107069 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:02.107112 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:02.107611 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:02.107836 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:02.107839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:02.108089 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:02.108285 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:02.108290 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.108931 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.108931 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:02.109751 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:02.109753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.110143 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.110144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:02.110263 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:02.110824 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:14:02.110832 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:02.112173 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:02.113827 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:02.113835 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:02.113838 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:02.114071 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:02.114259 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:02.114260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:02.114261 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:02.114370 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:02.114594 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:02.114601 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:02.114807 (+ 206us) client_negotiation.cc:769] Sending connection context
0924 23:14:02.114901 (+ 94us) client_negotiation.cc:240] Negotiation successful
0924 23:14:02.115067 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
I20260924 23:14:02.115803 29494 heartbeater.cc:347] Connected to a master server at 127.12.212.62:36023
I20260924 23:14:02.115900 29494 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:02.116070 29494 heartbeater.cc:511] Master 127.12.212.62:36023 requested a full tablet report, sending...
I20260924 23:14:02.116458 29035 ts_manager.cc:194] Registered new tserver with Master: 945c11a977764bfba378cba52cb70039 (127.12.212.3:44561)
I20260924 23:14:02.116881 29035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:39129
I20260924 23:14:02.120018 29498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:02.113600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33130 (local address 127.12.212.62:36023)
0924 23:14:02.113794 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:02.113796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:02.113803 (+ 7us) server_negotiation.cc:407] Connection header received
0924 23:14:02.113824 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:02.113824 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:02.113835 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:02.113889 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:02.114465 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.115037 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.116004 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.116227 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.116285 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:02.117596 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:02.117598 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:02.117598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:02.117607 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:02.118986 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:02.119334 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:02.119335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:02.119335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:02.119385 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:02.119608 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:02.119609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:02.119609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:02.119736 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:02.119849 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:02.119883 (+ 34us) server_negotiation.cc:299] Negotiation successful
0924 23:14:02.119943 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":127,"threads_started":1}
I20260924 23:14:02.120729 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 23:14:02.123564 29498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:02.121087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33146 (local address 127.12.212.62:36023)
0924 23:14:02.121133 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:02.121135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:02.121188 (+ 53us) server_negotiation.cc:407] Connection header received
0924 23:14:02.121273 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:02.121274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:02.121313 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:02.121368 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:14:02.121488 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.121966 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.122774 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.122974 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.123028 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:02.123183 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:14:02.123237 (+ 54us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0924 23:14:02.123495 (+ 258us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33146: token expired
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 23:14:02.123618 29498 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33146: token expired
W20260924 23:14:02.123718 28966 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.12.212.62:36023)
I20260924 23:14:02.130164 29498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:02.124011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33156 (local address 127.12.212.62:36023)
0924 23:14:02.124061 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:02.124062 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:02.124069 (+ 7us) server_negotiation.cc:407] Connection header received
0924 23:14:02.124089 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:02.124089 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:02.124099 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:02.124143 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:02.124595 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.125053 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.125872 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.126057 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.126111 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:02.127425 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:02.127426 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:02.127427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:02.127435 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:02.129023 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:02.129416 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:02.129417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:02.129418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:02.129475 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:02.129664 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:02.129665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:02.129665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:02.129780 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:02.129871 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:02.129977 (+ 106us) server_negotiation.cc:299] Negotiation successful
0924 23:14:02.130071 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 23:14:02.131412 29035 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:14:02.133100 29035 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:14:02.135546 29157 tablet_service.cc:1467] Processing CreateTablet for tablet 4962ca81dfaf435295a91e75f90a5256 (DEFAULT_TABLE table=test-table [id=04235053fec94362bfd1ff531df7fc7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:02.135775 29157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4962ca81dfaf435295a91e75f90a5256. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:02.137463 29115 tablet_bootstrap.cc:492] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Bootstrap starting.
I20260924 23:14:02.137774 29115 tablet_bootstrap.cc:654] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:02.138283 29115 tablet_bootstrap.cc:492] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e: No bootstrap required, opened a new log
I20260924 23:14:02.138343 29115 ts_tablet_manager.cc:1397] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260924 23:14:02.138684 29115 raft_consensus.cc:348] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } }
I20260924 23:14:02.138800 29115 raft_consensus.cc:374] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:02.138852 29115 raft_consensus.cc:729] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ec283fbe3cd4e0182faa7937a6a8f7e, State: Initialized, Role: FOLLOWER
I20260924 23:14:02.138895 29115 consensus_queue.cc:260] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } }
I20260924 23:14:02.138983 29115 ts_tablet_manager.cc:1428] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:02.139631 29230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:02.135155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:44289 (local address 127.0.0.1:38084)
0924 23:14:02.135267 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:02.135279 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:02.135331 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:02.136039 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:02.136040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:02.136041 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:02.136082 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:02.136083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.137094 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.137095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:02.138387 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:02.138390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.139360 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.139361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:02.139442 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:02.139460 (+ 18us) client_negotiation.cc:769] Sending connection context
0924 23:14:02.139514 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 23:14:02.139559 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260924 23:14:02.139675 29502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:02.135645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38084 (local address 127.12.212.2:44289)
0924 23:14:02.135829 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:02.135873 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:02.135884 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:14:02.135921 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:02.135924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:02.135938 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:02.136042 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:02.136126 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.136938 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.138563 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.139285 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.139353 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:02.139418 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:02.139541 (+ 123us) server_negotiation.cc:299] Negotiation successful
0924 23:14:02.139585 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":61,"threads_started":1}
I20260924 23:14:02.139726 29500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:02.135155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:44561 (local address 127.0.0.1:46164)
0924 23:14:02.135282 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:02.135290 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:02.135340 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:02.135870 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:02.135871 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:02.135874 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:02.135942 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:02.135944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.137231 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.137232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:02.138398 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:02.138399 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.139456 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.139457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:02.139527 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:02.139531 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:14:02.139587 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 23:14:02.139631 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":18,"thread_start_us":71,"threads_started":1}
I20260924 23:14:02.140079 29501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:02.135431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46164 (local address 127.12.212.3:44561)
0924 23:14:02.135625 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:02.135630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:02.135696 (+ 66us) server_negotiation.cc:407] Connection header received
0924 23:14:02.135741 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:02.135743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:02.135760 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:02.135850 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:02.136048 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.137152 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.138489 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.139279 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.139353 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:02.139418 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:02.139621 (+ 203us) server_negotiation.cc:299] Negotiation successful
0924 23:14:02.139668 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":100,"threads_started":1}
I20260924 23:14:02.140378 29295 tablet_service.cc:1467] Processing CreateTablet for tablet 4962ca81dfaf435295a91e75f90a5256 (DEFAULT_TABLE table=test-table [id=04235053fec94362bfd1ff531df7fc7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:02.140756 29295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4962ca81dfaf435295a91e75f90a5256. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:02.140662 29428 tablet_service.cc:1467] Processing CreateTablet for tablet 4962ca81dfaf435295a91e75f90a5256 (DEFAULT_TABLE table=test-table [id=04235053fec94362bfd1ff531df7fc7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:02.140964 29428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4962ca81dfaf435295a91e75f90a5256. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:02.143097 29386 tablet_bootstrap.cc:492] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039: Bootstrap starting.
I20260924 23:14:02.143903 29386 tablet_bootstrap.cc:654] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:02.144136 29386 log.cc:824] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:02.144310 29504 tablet_bootstrap.cc:492] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567: Bootstrap starting.
I20260924 23:14:02.144905 29504 tablet_bootstrap.cc:654] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:02.144965 29386 tablet_bootstrap.cc:492] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039: No bootstrap required, opened a new log
I20260924 23:14:02.145006 29386 ts_tablet_manager.cc:1397] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 23:14:02.145403 29504 log.cc:824] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:02.146063 29504 tablet_bootstrap.cc:492] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567: No bootstrap required, opened a new log
I20260924 23:14:02.146122 29504 ts_tablet_manager.cc:1397] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:14:02.146397 29386 raft_consensus.cc:348] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } }
I20260924 23:14:02.146536 29386 raft_consensus.cc:374] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:02.146559 29386 raft_consensus.cc:729] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945c11a977764bfba378cba52cb70039, State: Initialized, Role: FOLLOWER
I20260924 23:14:02.146663 29386 consensus_queue.cc:260] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } }
I20260924 23:14:02.146971 29386 ts_tablet_manager.cc:1428] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:14:02.147042 29494 heartbeater.cc:503] Master 127.12.212.62:36023 was elected leader, sending a full tablet report...
I20260924 23:14:02.147811 29504 raft_consensus.cc:348] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } }
I20260924 23:14:02.148028 29504 raft_consensus.cc:374] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:02.148080 29504 raft_consensus.cc:729] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e2fd14b901f4dabb87804e3ed361567, State: Initialized, Role: FOLLOWER
I20260924 23:14:02.148197 29504 consensus_queue.cc:260] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } }
I20260924 23:14:02.148486 29504 ts_tablet_manager.cc:1428] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260924 23:14:02.148526 29361 heartbeater.cc:503] Master 127.12.212.62:36023 was elected leader, sending a full tablet report...
W20260924 23:14:02.203145 29362 tablet.cc:2369] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:14:02.213423 29508 raft_consensus.cc:482] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:14:02.213562 29508 raft_consensus.cc:504] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } }
I20260924 23:14:02.213986 29508 leader_election.cc:288] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 945c11a977764bfba378cba52cb70039 (127.12.212.3:44561), 3ec283fbe3cd4e0182faa7937a6a8f7e (127.12.212.1:37997)
I20260924 23:14:02.217335 29363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:02.214149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:44561 (local address 127.12.212.2:44285)
0924 23:14:02.214240 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:02.214253 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:02.214298 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:02.214526 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:02.214527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:02.214529 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:02.214599 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:02.214602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.215324 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.215325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:02.216279 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:02.216282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.217070 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.217071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:02.217154 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:02.217158 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:14:02.217217 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 23:14:02.217260 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260924 23:14:02.217344 29501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:02.214307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:44285 (local address 127.12.212.3:44561)
0924 23:14:02.214354 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:02.214356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:02.214365 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:14:02.214393 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:02.214394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:02.214404 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:02.214502 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:02.214666 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.215256 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.216376 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.216987 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.217068 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:02.217078 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:02.217230 (+ 152us) server_negotiation.cc:299] Negotiation successful
0924 23:14:02.217269 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 23:14:02.217646 29231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:02.214245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:40427 (local address 127.12.212.1:37997)
0924 23:14:02.214292 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:02.214294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:02.214552 (+ 258us) server_negotiation.cc:407] Connection header received
0924 23:14:02.214654 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:02.214655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:02.214685 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:02.214781 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:02.214923 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.215677 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.216739 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.217197 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.217255 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:02.217346 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:02.217530 (+ 184us) server_negotiation.cc:299] Negotiation successful
0924 23:14:02.217582 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 23:14:02.217844 29509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:02.214155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:37997 (local address 127.12.212.2:40427)
0924 23:14:02.214491 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:02.214501 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:02.214576 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:02.214784 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:02.214785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:02.214787 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:02.214851 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:02.214853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.215760 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.215761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:02.216657 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:02.216660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.217379 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.217380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:02.217444 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:02.217449 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 23:14:02.217731 (+ 282us) client_negotiation.cc:240] Negotiation successful
0924 23:14:02.217776 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":187,"thread_start_us":91,"threads_started":1}
I20260924 23:14:02.217847 29448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4962ca81dfaf435295a91e75f90a5256" candidate_uuid: "3e2fd14b901f4dabb87804e3ed361567" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945c11a977764bfba378cba52cb70039" is_pre_election: true
I20260924 23:14:02.218031 29448 raft_consensus.cc:2454] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e2fd14b901f4dabb87804e3ed361567 in term 0.
I20260924 23:14:02.218075 29177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4962ca81dfaf435295a91e75f90a5256" candidate_uuid: "3e2fd14b901f4dabb87804e3ed361567" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" is_pre_election: true
I20260924 23:14:02.218241 29177 raft_consensus.cc:2454] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e2fd14b901f4dabb87804e3ed361567 in term 0.
I20260924 23:14:02.218279 29249 leader_election.cc:302] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e2fd14b901f4dabb87804e3ed361567, 945c11a977764bfba378cba52cb70039; no voters:
I20260924 23:14:02.218396 29508 raft_consensus.cc:2781] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:14:02.218451 29508 raft_consensus.cc:482] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:14:02.218470 29508 raft_consensus.cc:3037] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:02.219208 29508 raft_consensus.cc:504] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } }
I20260924 23:14:02.219343 29508 leader_election.cc:288] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [CANDIDATE]: Term 1 election: Requested vote from peers 945c11a977764bfba378cba52cb70039 (127.12.212.3:44561), 3ec283fbe3cd4e0182faa7937a6a8f7e (127.12.212.1:37997)
I20260924 23:14:02.219480 29177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4962ca81dfaf435295a91e75f90a5256" candidate_uuid: "3e2fd14b901f4dabb87804e3ed361567" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e"
I20260924 23:14:02.219513 29448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4962ca81dfaf435295a91e75f90a5256" candidate_uuid: "3e2fd14b901f4dabb87804e3ed361567" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945c11a977764bfba378cba52cb70039"
I20260924 23:14:02.219558 29177 raft_consensus.cc:3037] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:02.219607 29448 raft_consensus.cc:3037] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:02.220320 29177 raft_consensus.cc:2454] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e2fd14b901f4dabb87804e3ed361567 in term 1.
I20260924 23:14:02.220476 29248 leader_election.cc:302] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e2fd14b901f4dabb87804e3ed361567, 3ec283fbe3cd4e0182faa7937a6a8f7e; no voters:
I20260924 23:14:02.220558 29448 raft_consensus.cc:2454] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e2fd14b901f4dabb87804e3ed361567 in term 1.
I20260924 23:14:02.220583 29508 raft_consensus.cc:2781] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:02.220719 29508 raft_consensus.cc:686] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [term 1 LEADER]: Becoming Leader. State: Replica: 3e2fd14b901f4dabb87804e3ed361567, State: Running, Role: LEADER
I20260924 23:14:02.220817 29508 consensus_queue.cc:237] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } }
I20260924 23:14:02.221555 29035 catalog_manager.cc:5495] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e2fd14b901f4dabb87804e3ed361567 (127.12.212.2). New cstate: current_term: 1 leader_uuid: "3e2fd14b901f4dabb87804e3ed361567" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } health_report { overall_health: UNKNOWN } } }
I20260924 23:14:02.234723 29502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:02.231721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38090 (local address 127.12.212.2:44289)
0924 23:14:02.231793 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:02.231796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:02.231807 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:14:02.231858 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:02.231859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:02.231881 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:02.231962 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:14:02.232069 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.232876 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.233669 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:02.233984 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:02.234057 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:02.234305 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:14:02.234456 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:14:02.234540 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:02.234563 (+ 23us) server_negotiation.cc:299] Negotiation successful
0924 23:14:02.234587 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 23:14:02.236567 29448 raft_consensus.cc:1261] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039 [term 1 FOLLOWER]: Refusing update from remote peer 3e2fd14b901f4dabb87804e3ed361567: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:14:02.236613 29177 raft_consensus.cc:1261] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Refusing update from remote peer 3e2fd14b901f4dabb87804e3ed361567: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:14:02.236912 29508 consensus_queue.cc:1035] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [LEADER]: Connected to new peer: Peer: permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:14:02.237032 29510 consensus_queue.cc:1035] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:14:02.239156 29513 mvcc.cc:205] Tried to move back new op lower bound from 7333034566598545408 to 7333034566536998912. Current Snapshot: MvccSnapshot[applied={T|T < 7333034566598545408}]
I20260924 23:14:02.240257 29516 mvcc.cc:205] Tried to move back new op lower bound from 7333034566598545408 to 7333034566536998912. Current Snapshot: MvccSnapshot[applied={T|T < 7333034566598545408}]
I20260924 23:14:02.240693 29514 mvcc.cc:205] Tried to move back new op lower bound from 7333034566598545408 to 7333034566536998912. Current Snapshot: MvccSnapshot[applied={T|T < 7333034566598545408}]
I20260924 23:14:02.242218 29035 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:14:02.242323 29035 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:14:02.243104 29035 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc: Sending DeleteTablet for 3 replicas of tablet 4962ca81dfaf435295a91e75f90a5256
I20260924 23:14:02.243441 29295 tablet_service.cc:1514] Processing DeleteTablet for tablet 4962ca81dfaf435295a91e75f90a5256 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:02 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:38084
I20260924 23:14:02.243732 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 29073
I20260924 23:14:02.243846 29428 tablet_service.cc:1514] Processing DeleteTablet for tablet 4962ca81dfaf435295a91e75f90a5256 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:02 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:46164
I20260924 23:14:02.245479 29523 tablet_replica.cc:330] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039: stopping tablet replica
I20260924 23:14:02.245594 29523 raft_consensus.cc:2229] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:14:02.245700 29523 raft_consensus.cc:2258] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:14:02.246178 29523 ts_tablet_manager.cc:1905] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:14:02.247874 29522 tablet_replica.cc:330] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567: stopping tablet replica
I20260924 23:14:02.248016 29522 raft_consensus.cc:2229] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:14:02.248133 29522 raft_consensus.cc:2258] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:14:02.248699 29522 ts_tablet_manager.cc:1905] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 23:14:02.249989 29019 connection.cc:535] client connection to 127.12.212.1:37997 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 23:14:02.250111 29019 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 23:14:02.250135 29523 ts_tablet_manager.cc:1918] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:14:02.250192 29523 log.cc:1196] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/4962ca81dfaf435295a91e75f90a5256
I20260924 23:14:02.250411 29523 ts_tablet_manager.cc:1939] T 4962ca81dfaf435295a91e75f90a5256 P 945c11a977764bfba378cba52cb70039: Deleting consensus metadata
I20260924 23:14:02.250900 29020 catalog_manager.cc:4841] TS 945c11a977764bfba378cba52cb70039 (127.12.212.3:44561): tablet 4962ca81dfaf435295a91e75f90a5256 (table test-table [id=04235053fec94362bfd1ff531df7fc7e]) successfully deleted
I20260924 23:14:02.250949 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 29227
I20260924 23:14:02.252583 29500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:02.252419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:37997 (local address 127.0.0.1:46730)
0924 23:14:02.252490 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:02.252544 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.1:37997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260924 23:14:02.252720 29019 catalog_manager.cc:4584] TS 3ec283fbe3cd4e0182faa7937a6a8f7e (127.12.212.1:37997): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4962ca81dfaf435295a91e75f90a5256: Network error: Client connection negotiation failed: client connection to 127.12.212.1:37997: connect: Connection refused (error 111)
I20260924 23:14:02.255827 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 29364
I20260924 23:14:02.255847 29500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:02.255728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:44289 (local address 127.0.0.1:38106)
0924 23:14:02.255780 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:02.255808 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.2:44289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 23:14:02.260437 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 29002
I20260924 23:14:22.265291 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:36023
--webserver_interface=127.12.212.62
--webserver_port=35835
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:36023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:22.346884 29525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:22.347124 29525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:22.347149 29525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:22.348429 29525 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 23:14:22.348474 29525 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 23:14:22.348485 29525 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 23:14:22.348495 29525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:14:22.348507 29525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:22.348519 29525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:22.348531 29525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:14:22.348541 29525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:14:22.350087 29525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:36023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:36023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=35835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:22.350337 29525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:22.350541 29525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:22.356432 29533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:22.356432 29530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:22.356544 29531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:22.356639 29525 server_base.cc:1034] running on GCE node
I20260924 23:14:22.357007 29525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:22.357409 29525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:22.358593 29525 hybrid_clock.cc:648] HybridClock initialized: now 1790291662358579 us; error 33 us; skew 500 ppm
Sep 24 23:14:22 dist-test-slave-m0z6 krb5kdc[28456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291662, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:22.365999 29525 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:14:22.366498 29525 webserver.cc:460] Webserver started at http://127.12.212.62:35835/ using document root <none> and password file <none>
I20260924 23:14:22.366778 29525 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:22.366833 29525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:22.368294 29525 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:22.368955 29540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:22.369091 29525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:14:22.369189 29525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "032c6e3e5d754be1814dc735864864dc"
format_stamp: "Formatted at 2026-09-24 23:13:40 on dist-test-slave-m0z6"
server_key: "6f4b5b4eb1f2b12c8ce70ca717554283"
server_key_iv: "8db77c7fb2ce2aedbc222f834e64f712"
server_key_version: "encryptionkey@0"
I20260924 23:14:22.369542 29525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:22.380725 29525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:22.382839 29525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:22.382993 29525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:22.386655 29525 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:36023
I20260924 23:14:22.386745 29592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:36023 every 8 connection(s)
I20260924 23:14:22.387087 29525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:14:22.388562 29593 sys_catalog.cc:263] Verifying existing consensus state
I20260924 23:14:22.389190 29593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc: Bootstrap starting.
I20260924 23:14:22.390339 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 29525
I20260924 23:14:22.390589 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:37997
--local_ip_for_outbound_sockets=127.12.212.1
--tserver_master_addrs=127.12.212.62:36023
--webserver_port=35621
--webserver_interface=127.12.212.1
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 23:14:22.398101 29593 log.cc:824] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:22.400718 29593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 23:14:22.401057 29593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc: Bootstrap complete.
I20260924 23:14:22.403347 29593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032c6e3e5d754be1814dc735864864dc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36023 } }
I20260924 23:14:22.403604 29593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 032c6e3e5d754be1814dc735864864dc, State: Initialized, Role: FOLLOWER
I20260924 23:14:22.403764 29593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032c6e3e5d754be1814dc735864864dc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36023 } }
I20260924 23:14:22.403839 29593 raft_consensus.cc:388] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:14:22.403884 29593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:14:22.403935 29593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 2 FOLLOWER]: Advancing to term 3
I20260924 23:14:22.405102 29593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032c6e3e5d754be1814dc735864864dc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36023 } }
I20260924 23:14:22.405222 29593 leader_election.cc:302] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 032c6e3e5d754be1814dc735864864dc; no voters:
I20260924 23:14:22.405411 29593 leader_election.cc:288] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [CANDIDATE]: Term 3 election: Requested vote from peers
I20260924 23:14:22.405464 29597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 3 FOLLOWER]: Leader election won for term 3
I20260924 23:14:22.405604 29597 raft_consensus.cc:686] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [term 3 LEADER]: Becoming Leader. State: Replica: 032c6e3e5d754be1814dc735864864dc, State: Running, Role: LEADER
I20260924 23:14:22.405690 29593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:14:22.405695 29597 consensus_queue.cc:237] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032c6e3e5d754be1814dc735864864dc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36023 } }
I20260924 23:14:22.406164 29599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "032c6e3e5d754be1814dc735864864dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032c6e3e5d754be1814dc735864864dc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36023 } } }
I20260924 23:14:22.406260 29599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:22.406417 29598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 032c6e3e5d754be1814dc735864864dc. Latest consensus state: current_term: 3 leader_uuid: "032c6e3e5d754be1814dc735864864dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032c6e3e5d754be1814dc735864864dc" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 36023 } } }
I20260924 23:14:22.406461 29606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:14:22.406463 29598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:22.407394 29606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:14:22.407493 29606 catalog_manager.cc:1259] Loaded cluster ID: f1c9f08f4aba4985847c328c9c70611c
I20260924 23:14:22.407516 29606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:14:22.408631 29606 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:14:22.408828 29606 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc: Loaded TSK: 0
I20260924 23:14:22.408877 29606 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 032c6e3e5d754be1814dc735864864dc: Loaded TSK: 1
F20260924 23:14:22.409101 29606 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f76a175b9ad google::LogMessage::Fail() at ??:0
@ 0x7f76a175e88a google::LogMessage::SendToLog() at ??:0
@ 0x7f76a175b4de google::LogMessage::Flush() at ??:0
@ 0x7f76a175ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f76a3ec8104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f76a3ec5d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f76a96b51cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f76a96b548e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f76a96b5594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f76a967bc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f76a9685ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f76a36290d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f76a361e954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f76a44826db start_thread at ??:0
@ 0x7f76a025961f clone at ??:0
W20260924 23:14:22.490943 29595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:22.491150 29595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:22.491178 29595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:22.492378 29595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:22.492424 29595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:22.492450 29595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:14:22.493942 29595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:37997
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=35621
--tserver_master_addrs=127.12.212.62:36023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:22.494179 29595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:22.494405 29595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:22.499307 29637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:22.499328 29634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:22.499491 29595 server_base.cc:1034] running on GCE node
W20260924 23:14:22.499601 29635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:22.499840 29595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:22.500074 29595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:22.501291 29595 hybrid_clock.cc:648] HybridClock initialized: now 1790291662501257 us; error 59 us; skew 500 ppm
Sep 24 23:14:22 dist-test-slave-m0z6 krb5kdc[28456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291662, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:22.503724 29595 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:14:22.504195 29595 webserver.cc:460] Webserver started at http://127.12.212.1:35621/ using document root <none> and password file <none>
I20260924 23:14:22.504426 29595 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:22.504468 29595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:22.505839 29595 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:22.506493 29644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:22.506721 29595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:22.506820 29595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e"
format_stamp: "Formatted at 2026-09-24 23:13:41 on dist-test-slave-m0z6"
server_key: "321f561106d05edd2255cbca68661288"
server_key_iv: "f67cfbfe18b7788d80ac5f9edc874459"
server_key_version: "encryptionkey@0"
I20260924 23:14:22.507179 29595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:22.523741 29595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:22.526511 29595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:22.526684 29595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:22.527042 29595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:22.527506 29651 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 23:14:22.529503 29595 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 23:14:22.529556 29595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 23:14:22.529640 29595 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 23:14:22.530346 29595 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 23:14:22.530381 29595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 23:14:22.530481 29651 tablet_bootstrap.cc:492] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Bootstrap starting.
I20260924 23:14:22.537106 29595 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:37997
I20260924 23:14:22.537518 29595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:14:22.541468 29651 log.cc:824] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:22.541972 29758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:37997 every 8 connection(s)
I20260924 23:14:22.544670 29651 tablet_bootstrap.cc:492] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 23:14:22.544992 29651 tablet_bootstrap.cc:492] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Bootstrap complete.
I20260924 23:14:22.546053 29651 ts_tablet_manager.cc:1397] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.004s
I20260924 23:14:22.546389 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 29595
I20260924 23:14:22.546626 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:44289
--local_ip_for_outbound_sockets=127.12.212.2
--tserver_master_addrs=127.12.212.62:36023
--webserver_port=37241
--webserver_interface=127.12.212.2
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 23:14:22.547541 29651 raft_consensus.cc:348] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } }
I20260924 23:14:22.547895 29651 raft_consensus.cc:729] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ec283fbe3cd4e0182faa7937a6a8f7e, State: Initialized, Role: FOLLOWER
I20260924 23:14:22.548045 29651 consensus_queue.cc:260] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } }
I20260924 23:14:22.548333 29651 ts_tablet_manager.cc:1428] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
W20260924 23:14:22.615613 29761 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:14:22.538185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:60677)
0924 23:14:22.538386 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:22.538433 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:22.539003 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:22.615408 (+ 76405us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":83,"threads_started":1}
W20260924 23:14:22.615875 29759 heartbeater.cc:644] Failed to heartbeat to 127.12.212.62:36023 (0 consecutive failures): Network error: Failed to ping master at 127.12.212.62:36023: Client connection negotiation failed: client connection to 127.12.212.62:36023: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 23:14:22.616232 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:22.616116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:54375)
0924 23:14:22.616164 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:22.616207 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260924 23:14:22.616705 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:22.616558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:43601)
0924 23:14:22.616651 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:22.616683 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260924 23:14:22.616937 29759 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 23:14:22.647420 29763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:22.647648 29763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:22.647675 29763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:22.648890 29763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:22.648936 29763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:22.648962 29763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:14:22.650523 29763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:44289
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=37241
--tserver_master_addrs=127.12.212.62:36023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:22.650779 29763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:22.650982 29763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:22.655988 29772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:22.655894 29770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:22.655894 29769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:22.656090 29763 server_base.cc:1034] running on GCE node
I20260924 23:14:22.656595 29763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:22.656857 29763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:22.658010 29763 hybrid_clock.cc:648] HybridClock initialized: now 1790291662657999 us; error 29 us; skew 500 ppm
Sep 24 23:14:22 dist-test-slave-m0z6 krb5kdc[28456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291662, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:22.660514 29763 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:14:22.660985 29763 webserver.cc:460] Webserver started at http://127.12.212.2:37241/ using document root <none> and password file <none>
I20260924 23:14:22.661190 29763 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:22.661232 29763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:22.662516 29763 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:22.663251 29779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:22.663435 29763 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 23:14:22.663516 29763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "3e2fd14b901f4dabb87804e3ed361567"
format_stamp: "Formatted at 2026-09-24 23:13:41 on dist-test-slave-m0z6"
server_key: "b6da2d5bc2f95513db7d4e3135ab58f8"
server_key_iv: "7113fc632277eb922a16e5e5a4f55f4e"
server_key_version: "encryptionkey@0"
I20260924 23:14:22.663892 29763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:22.681341 29763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:22.683530 29763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:22.683668 29763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:22.683895 29763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:22.684382 29786 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260924 23:14:22.685873 29786 ts_tablet_manager.cc:1852] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260924 23:14:22.686153 29786 ts_tablet_manager.cc:1905] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:14:22.688583 29786 ts_tablet_manager.cc:1918] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:14:22.688680 29786 log.cc:1196] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/4962ca81dfaf435295a91e75f90a5256
I20260924 23:14:22.688973 29786 ts_tablet_manager.cc:1939] T 4962ca81dfaf435295a91e75f90a5256 P 3e2fd14b901f4dabb87804e3ed361567: Deleting consensus metadata
I20260924 23:14:22.689186 29763 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260924 23:14:22.689244 29763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260924 23:14:22.689282 29763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:22.689309 29763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:22.695147 29763 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:44289
I20260924 23:14:22.695216 29893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:44289 every 8 connection(s)
I20260924 23:14:22.695509 29763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:14:22.696501 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:22.696138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:37045)
0924 23:14:22.696293 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:22.696367 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
W20260924 23:14:22.696666 29894 heartbeater.cc:644] Failed to heartbeat to 127.12.212.62:36023 (0 consecutive failures): Network error: Failed to ping master at 127.12.212.62:36023: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
I20260924 23:14:22.696904 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:22.696824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:40121)
0924 23:14:22.696865 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:22.696889 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260924 23:14:22.697162 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:22.697106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:51725)
0924 23:14:22.697137 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:22.697153 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260924 23:14:22.697261 29894 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 23:14:22.702685 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 29763
I20260924 23:14:22.703016 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:44561
--local_ip_for_outbound_sockets=127.12.212.3
--tserver_master_addrs=127.12.212.62:36023
--webserver_port=44229
--webserver_interface=127.12.212.3
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:22.783879 29897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:22.784104 29897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:22.784127 29897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:22.785297 29897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:22.785339 29897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:22.785364 29897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:14:22.786855 29897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:44561
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=44229
--tserver_master_addrs=127.12.212.62:36023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:22.787084 29897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:22.787308 29897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:22.792340 29903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:22.792332 29905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:22.792498 29902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:22.792619 29897 server_base.cc:1034] running on GCE node
I20260924 23:14:22.792807 29897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:22.793066 29897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:22.794251 29897 hybrid_clock.cc:648] HybridClock initialized: now 1790291662794236 us; error 30 us; skew 500 ppm
Sep 24 23:14:22 dist-test-slave-m0z6 krb5kdc[28456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291662, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:22.796667 29897 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:14:22.797066 29897 webserver.cc:460] Webserver started at http://127.12.212.3:44229/ using document root <none> and password file <none>
I20260924 23:14:22.797261 29897 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:22.797302 29897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:22.798590 29897 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:22.799242 29912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:22.799432 29897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 23:14:22.799523 29897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "945c11a977764bfba378cba52cb70039"
format_stamp: "Formatted at 2026-09-24 23:13:41 on dist-test-slave-m0z6"
server_key: "da5099f248bea974ea6205ae8a25d79d"
server_key_iv: "a4f99f80be8e8629036eb99e4c39a022"
server_key_version: "encryptionkey@0"
I20260924 23:14:22.799924 29897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:22.810279 29897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:22.812651 29897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:22.812811 29897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:22.813050 29897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:22.813359 29897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:22.813393 29897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:22.813422 29897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:22.813439 29897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:22.819103 29897 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:44561
I20260924 23:14:22.819149 30025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:44561 every 8 connection(s)
I20260924 23:14:22.819489 29897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:14:22.820535 30028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:22.820140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:54525)
0924 23:14:22.820318 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:22.820396 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":64,"threads_started":1}
W20260924 23:14:22.820909 30026 heartbeater.cc:644] Failed to heartbeat to 127.12.212.62:36023 (0 consecutive failures): Network error: Failed to ping master at 127.12.212.62:36023: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
I20260924 23:14:22.821143 30028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:22.821064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:59035)
0924 23:14:22.821103 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:22.821127 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260924 23:14:22.821342 30028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:22.821287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:54033)
0924 23:14:22.821309 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:22.821329 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260924 23:14:22.821427 30026 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 23:14:22.828037 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 29897
I20260924 23:14:23.617585 30030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:23.617308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:36837)
0924 23:14:23.617475 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:23.617519 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260924 23:14:23.697846 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:23.697618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:50007)
0924 23:14:23.697740 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:23.697807 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":52,"threads_started":1}
I20260924 23:14:23.821997 30032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:23.821785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:53991)
0924 23:14:23.821909 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:23.821945 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
I20260924 23:14:24.045333 30033 raft_consensus.cc:482] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:14:24.045468 30033 raft_consensus.cc:504] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } }
I20260924 23:14:24.045753 30033 leader_election.cc:288] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 945c11a977764bfba378cba52cb70039 (127.12.212.3:44561), 3e2fd14b901f4dabb87804e3ed361567 (127.12.212.2:44289)
Sep 24 23:14:24 dist-test-slave-m0z6 krb5kdc[28456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291662, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.3@KRBTEST.COM
Sep 24 23:14:24 dist-test-slave-m0z6 krb5kdc[28456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291662, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.2@KRBTEST.COM
I20260924 23:14:24.057114 30030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:24.045951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:44561 (local address 127.12.212.1:56529)
0924 23:14:24.046012 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:24.046022 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:24.046076 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:24.046935 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:24.046938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:24.047353 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:24.047549 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:24.047555 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:24.048658 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:24.048659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:24.049484 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:24.049486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:24.050179 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:24.050180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:24.050303 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:24.050962 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:14:24.050966 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:24.052858 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:24.055132 (+ 2274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:24.055141 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:24.055143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:24.055446 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:24.055775 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:24.055776 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:24.055776 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:24.055924 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:24.056431 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:24.056448 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:24.056633 (+ 185us) client_negotiation.cc:769] Sending connection context
0924 23:14:24.056756 (+ 123us) client_negotiation.cc:240] Negotiation successful
0924 23:14:24.056969 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":54}
I20260924 23:14:24.057126 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:24.046042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:56529 (local address 127.12.212.3:44561)
0924 23:14:24.046187 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:24.046201 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:24.046229 (+ 28us) server_negotiation.cc:407] Connection header received
0924 23:14:24.046666 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:24.046668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:24.046764 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:24.046897 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:24.047676 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:24.048560 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:24.049575 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:24.050076 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:24.050190 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:24.053028 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:24.053033 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:24.053033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:24.053045 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:24.055028 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:24.055553 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:24.055572 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:24.055572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:24.055699 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:24.055991 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:24.055992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:24.055992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:24.056337 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:24.056436 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:24.056854 (+ 418us) server_negotiation.cc:299] Negotiation successful
0924 23:14:24.057004 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":56,"threads_started":1}
I20260924 23:14:24.057128 30036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:24.046089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:36481 (local address 127.12.212.2:44289)
0924 23:14:24.046297 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:24.046307 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:24.046326 (+ 19us) server_negotiation.cc:407] Connection header received
0924 23:14:24.046846 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:24.046850 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:24.046904 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:24.047045 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:24.047663 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:24.048536 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:24.049574 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:24.050107 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:24.050240 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:24.053394 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:24.053399 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:24.053399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:24.053410 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:24.055251 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:24.055662 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:24.055665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:24.055666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:24.055786 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:24.055988 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:24.055988 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:24.055989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:24.056402 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:24.056473 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:24.056849 (+ 376us) server_negotiation.cc:299] Negotiation successful
0924 23:14:24.057004 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":49,"threads_started":1}
I20260924 23:14:24.057123 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:24.045978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:44289 (local address 127.12.212.1:36481)
0924 23:14:24.046129 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:24.046139 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:24.046195 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:24.047029 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:24.047030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:24.047351 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:24.047549 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:24.047555 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:24.048641 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:24.048642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:24.049483 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:24.049486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:24.050211 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:24.050212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:24.050304 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:24.050925 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:14:24.050936 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:24.053251 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:24.055390 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:24.055391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:24.055391 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:24.055575 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:24.055855 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:24.055855 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:24.055856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:24.055924 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:24.056479 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:24.056480 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:24.056633 (+ 153us) client_negotiation.cc:769] Sending connection context
0924 23:14:24.056756 (+ 123us) client_negotiation.cc:240] Negotiation successful
0924 23:14:24.057036 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":164,"thread_start_us":77,"threads_started":1}
W20260924 23:14:24.058295 29646 leader_election.cc:341] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 945c11a977764bfba378cba52cb70039 (127.12.212.3:44561): Not found: Tablet not found: 4962ca81dfaf435295a91e75f90a5256
W20260924 23:14:24.058445 29645 leader_election.cc:341] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3e2fd14b901f4dabb87804e3ed361567 (127.12.212.2:44289): Not found: Tablet not found: 4962ca81dfaf435295a91e75f90a5256
I20260924 23:14:24.058509 29645 leader_election.cc:302] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ec283fbe3cd4e0182faa7937a6a8f7e; no voters: 3e2fd14b901f4dabb87804e3ed361567, 945c11a977764bfba378cba52cb70039
I20260924 23:14:24.058642 30033 raft_consensus.cc:2726] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 23:14:24.618436 30037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:24.618169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:47449)
0924 23:14:24.618338 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:24.618388 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":69,"threads_started":1}
I20260924 23:14:24.698582 30038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:24.698353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:44673)
0924 23:14:24.698493 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:24.698539 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":60,"threads_started":1}
I20260924 23:14:24.822675 30039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:24.822424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:46143)
0924 23:14:24.822575 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:24.822622 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":60,"threads_started":1}
I20260924 23:14:25.619280 30040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:25.619002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:59745)
0924 23:14:25.619165 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:25.619220 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":74,"threads_started":1}
I20260924 23:14:25.699316 30041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:25.699098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:55303)
0924 23:14:25.699225 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:25.699271 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":59,"threads_started":1}
I20260924 23:14:25.793686 30042 raft_consensus.cc:482] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:14:25.793776 30042 raft_consensus.cc:504] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } }
I20260924 23:14:25.793931 30042 leader_election.cc:288] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 945c11a977764bfba378cba52cb70039 (127.12.212.3:44561), 3e2fd14b901f4dabb87804e3ed361567 (127.12.212.2:44289)
W20260924 23:14:25.794353 29646 leader_election.cc:341] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 945c11a977764bfba378cba52cb70039 (127.12.212.3:44561): Not found: Tablet not found: 4962ca81dfaf435295a91e75f90a5256
W20260924 23:14:25.794446 29645 leader_election.cc:341] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3e2fd14b901f4dabb87804e3ed361567 (127.12.212.2:44289): Not found: Tablet not found: 4962ca81dfaf435295a91e75f90a5256
I20260924 23:14:25.794520 29645 leader_election.cc:302] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ec283fbe3cd4e0182faa7937a6a8f7e; no voters: 3e2fd14b901f4dabb87804e3ed361567, 945c11a977764bfba378cba52cb70039
I20260924 23:14:25.794595 30042 raft_consensus.cc:2726] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 23:14:25.823463 30043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:25.823204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:43929)
0924 23:14:25.823370 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:25.823419 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":76,"threads_started":1}
I20260924 23:14:26.620236 30044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:26.619928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:50369)
0924 23:14:26.620085 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:26.620134 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
I20260924 23:14:26.700021 30045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:26.699794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:35035)
0924 23:14:26.699930 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:26.699975 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
I20260924 23:14:26.824198 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:26.823966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:51493)
0924 23:14:26.824109 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:26.824150 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
I20260924 23:14:27.621022 30047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:27.620791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:42795)
0924 23:14:27.620933 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:27.620974 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":62,"threads_started":1}
I20260924 23:14:27.700815 30048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:27.700583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:55199)
0924 23:14:27.700725 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:27.700772 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":60,"threads_started":1}
I20260924 23:14:27.824923 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:27.824713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:38083)
0924 23:14:27.824836 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:27.824878 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":58,"threads_started":1}
I20260924 23:14:28.153864 30050 raft_consensus.cc:482] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:14:28.153966 30050 raft_consensus.cc:504] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } }
I20260924 23:14:28.154158 30050 leader_election.cc:288] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 945c11a977764bfba378cba52cb70039 (127.12.212.3:44561), 3e2fd14b901f4dabb87804e3ed361567 (127.12.212.2:44289)
W20260924 23:14:28.154580 29646 leader_election.cc:341] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 945c11a977764bfba378cba52cb70039 (127.12.212.3:44561): Not found: Tablet not found: 4962ca81dfaf435295a91e75f90a5256
W20260924 23:14:28.154729 29645 leader_election.cc:341] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3e2fd14b901f4dabb87804e3ed361567 (127.12.212.2:44289): Not found: Tablet not found: 4962ca81dfaf435295a91e75f90a5256
I20260924 23:14:28.154798 29645 leader_election.cc:302] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ec283fbe3cd4e0182faa7937a6a8f7e; no voters: 3e2fd14b901f4dabb87804e3ed361567, 945c11a977764bfba378cba52cb70039
I20260924 23:14:28.154860 30050 raft_consensus.cc:2726] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 23:14:28.621793 30051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:28.621545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:41999)
0924 23:14:28.621690 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:28.621741 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":66,"threads_started":1}
I20260924 23:14:28.701538 30052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:28.701328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:45727)
0924 23:14:28.701448 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:28.701491 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260924 23:14:28.825657 30053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:28.825416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:45385)
0924 23:14:28.825567 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:28.825603 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":71,"threads_started":1}
I20260924 23:14:29.622534 30054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:29.622291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:42979)
0924 23:14:29.622441 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:29.622485 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":63,"threads_started":1}
I20260924 23:14:29.702428 30055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:29.702099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:42953)
0924 23:14:29.702328 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:29.702380 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":66,"threads_started":1}
I20260924 23:14:29.826273 30056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:29.826062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:34591)
0924 23:14:29.826191 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:29.826230 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260924 23:14:30.623287 30057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:30.623037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:40345)
0924 23:14:30.623156 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:30.623208 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260924 23:14:30.703177 30058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:30.702947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:49133)
0924 23:14:30.703088 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:30.703131 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":64,"threads_started":1}
I20260924 23:14:30.827028 30059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:30.826822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:60731)
0924 23:14:30.826939 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:30.826979 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260924 23:14:31.624007 30060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:31.623748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:44873)
0924 23:14:31.623915 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:31.623960 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":76,"threads_started":1}
I20260924 23:14:31.703907 30061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:31.703694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:52749)
0924 23:14:31.703832 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:31.703865 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260924 23:14:31.827811 30062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:31.827552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:41875)
0924 23:14:31.827707 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:31.827765 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
I20260924 23:14:32.463428 30063 raft_consensus.cc:482] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:14:32.463521 30063 raft_consensus.cc:504] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } }
I20260924 23:14:32.463788 30063 leader_election.cc:288] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 945c11a977764bfba378cba52cb70039 (127.12.212.3:44561), 3e2fd14b901f4dabb87804e3ed361567 (127.12.212.2:44289)
W20260924 23:14:32.464243 29646 leader_election.cc:341] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 945c11a977764bfba378cba52cb70039 (127.12.212.3:44561): Not found: Tablet not found: 4962ca81dfaf435295a91e75f90a5256
W20260924 23:14:32.464381 29645 leader_election.cc:341] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3e2fd14b901f4dabb87804e3ed361567 (127.12.212.2:44289): Not found: Tablet not found: 4962ca81dfaf435295a91e75f90a5256
I20260924 23:14:32.464421 29645 leader_election.cc:302] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ec283fbe3cd4e0182faa7937a6a8f7e; no voters: 3e2fd14b901f4dabb87804e3ed361567, 945c11a977764bfba378cba52cb70039
I20260924 23:14:32.464491 30063 raft_consensus.cc:2726] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 23:14:32.624816 30064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:32.624595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:52199)
0924 23:14:32.624715 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:32.624755 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
I20260924 23:14:32.704613 30065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:32.704375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:47503)
0924 23:14:32.704520 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:32.704567 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":64,"threads_started":1}
I20260924 23:14:32.828519 30066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:32.828303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:56819)
0924 23:14:32.828417 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:32.828458 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260924 23:14:33.625576 30067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:33.625311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:40835)
0924 23:14:33.625470 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:33.625515 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260924 23:14:33.705413 30068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:33.705124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:55511)
0924 23:14:33.705314 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:33.705364 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":77,"threads_started":1}
I20260924 23:14:33.829340 30069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:33.829018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:60013)
0924 23:14:33.829230 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:33.829279 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":98,"threads_started":1}
I20260924 23:14:34.626405 30070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:34.626148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:39683)
0924 23:14:34.626301 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:34.626353 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260924 23:14:34.706095 30071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:34.705876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:33851)
0924 23:14:34.706008 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:34.706049 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260924 23:14:34.830085 30072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:34.829863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:39785)
0924 23:14:34.829992 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:34.830033 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
I20260924 23:14:35.147614 30073 raft_consensus.cc:482] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:14:35.147707 30073 raft_consensus.cc:504] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945c11a977764bfba378cba52cb70039" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 44561 } } peers { permanent_uuid: "3e2fd14b901f4dabb87804e3ed361567" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44289 } } peers { permanent_uuid: "3ec283fbe3cd4e0182faa7937a6a8f7e" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37997 } }
I20260924 23:14:35.147836 30073 leader_election.cc:288] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 945c11a977764bfba378cba52cb70039 (127.12.212.3:44561), 3e2fd14b901f4dabb87804e3ed361567 (127.12.212.2:44289)
W20260924 23:14:35.148259 29646 leader_election.cc:341] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 945c11a977764bfba378cba52cb70039 (127.12.212.3:44561): Not found: Tablet not found: 4962ca81dfaf435295a91e75f90a5256
W20260924 23:14:35.148362 29645 leader_election.cc:341] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3e2fd14b901f4dabb87804e3ed361567 (127.12.212.2:44289): Not found: Tablet not found: 4962ca81dfaf435295a91e75f90a5256
I20260924 23:14:35.148406 29645 leader_election.cc:302] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ec283fbe3cd4e0182faa7937a6a8f7e; no voters: 3e2fd14b901f4dabb87804e3ed361567, 945c11a977764bfba378cba52cb70039
I20260924 23:14:35.148468 30073 raft_consensus.cc:2726] T 4962ca81dfaf435295a91e75f90a5256 P 3ec283fbe3cd4e0182faa7937a6a8f7e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 23:14:35.627238 30074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:35.626990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:59887)
0924 23:14:35.627141 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:35.627193 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260924 23:14:35.706768 30075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:35.706546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:53679)
0924 23:14:35.706656 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:35.706712 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":65,"threads_started":1}
I20260924 23:14:35.830858 30076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:35.830604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:43723)
0924 23:14:35.830765 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:35.830811 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":75,"threads_started":1}
I20260924 23:14:36.628151 30077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:36.627855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:46081)
0924 23:14:36.628045 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:36.628103 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":84,"threads_started":1}
I20260924 23:14:36.707506 30078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:36.707230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:44607)
0924 23:14:36.707388 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:36.707439 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":69,"threads_started":1}
I20260924 23:14:36.831512 30079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:36.831269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:47059)
0924 23:14:36.831427 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:36.831466 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":66,"threads_started":1}
I20260924 23:14:37.629033 30080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:37.628735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.1:40749)
0924 23:14:37.628924 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:37.628975 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":86,"threads_started":1}
I20260924 23:14:37.708338 30081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:37.708115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.2:45317)
0924 23:14:37.708258 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:37.708291 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
I20260924 23:14:37.832365 30082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:37.832114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:36023 (local address 127.12.212.3:59527)
0924 23:14:37.832268 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:37.832314 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.62:36023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":63,"threads_started":1}
I20260924 23:14:37.835137 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 29595
I20260924 23:14:37.839479 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 29763
I20260924 23:14:37.843487 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 29897
2026-09-24T23:14:37Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58211 ms)
[----------] 30 tests from SecurityITest (233831 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:14:37 dist-test-slave-m0z6 krb5kdc[30085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:14:37 dist-test-slave-m0z6 krb5kdc[30085](info): set up 2 sockets
Sep 24 23:14:37 dist-test-slave-m0z6 krb5kdc[30085](info): commencing operation
krb5kdc: starting...
W20260924 23:14:38.802312 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:14:38 dist-test-slave-m0z6 krb5kdc[30085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:14:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:14:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:14:38.924084 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:42983
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:42983
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:39.003912 30098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:39.004135 30098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:39.004159 30098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:39.005417 30098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:14:39.005471 30098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:39.005486 30098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:39.005496 30098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:14:39.005508 30098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:14:39.007148 30098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:42983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:42983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:39.007411 30098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:39.007643 30098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:39.012497 30104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:39.012501 30106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:39.012501 30103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:39.012764 30098 server_base.cc:1034] running on GCE node
I20260924 23:14:39.013062 30098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:39.013355 30098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:39.014534 30098 hybrid_clock.cc:648] HybridClock initialized: now 1790291679014505 us; error 48 us; skew 500 ppm
Sep 24 23:14:39 dist-test-slave-m0z6 krb5kdc[30085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291679, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:39.017148 30098 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:14:39.017604 30098 webserver.cc:460] Webserver started at http://127.12.212.62:38773/ using document root <none> and password file <none>
I20260924 23:14:39.017838 30098 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:39.017889 30098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:39.018000 30098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:39.019155 30098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "b9fc85a526304f33b0d01e7f15387396"
format_stamp: "Formatted at 2026-09-24 23:14:39 on dist-test-slave-m0z6"
server_key: "da0e2303388fd6f951da63efc7b2a46c"
server_key_iv: "136526b809371df86b27e5b7de52b4fe"
server_key_version: "encryptionkey@0"
I20260924 23:14:39.019493 30098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "b9fc85a526304f33b0d01e7f15387396"
format_stamp: "Formatted at 2026-09-24 23:14:39 on dist-test-slave-m0z6"
server_key: "da0e2303388fd6f951da63efc7b2a46c"
server_key_iv: "136526b809371df86b27e5b7de52b4fe"
server_key_version: "encryptionkey@0"
I20260924 23:14:39.020754 30098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:39.021459 30113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:39.021636 30098 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:14:39.021742 30098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "b9fc85a526304f33b0d01e7f15387396"
format_stamp: "Formatted at 2026-09-24 23:14:39 on dist-test-slave-m0z6"
server_key: "da0e2303388fd6f951da63efc7b2a46c"
server_key_iv: "136526b809371df86b27e5b7de52b4fe"
server_key_version: "encryptionkey@0"
I20260924 23:14:39.021801 30098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:39.056643 30098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:39.059199 30098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:39.059319 30098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:39.063131 30098 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:42983
I20260924 23:14:39.063172 30165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:42983 every 8 connection(s)
I20260924 23:14:39.063524 30098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:14:39.064379 30166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:39.066685 30166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396: Bootstrap starting.
I20260924 23:14:39.067287 30166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:39.067539 30166 log.cc:824] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:39.068195 30166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396: No bootstrap required, opened a new log
I20260924 23:14:39.069216 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 30098
I20260924 23:14:39.069324 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:14:39.069514 13136 external_mini_cluster.cc:1428] Setting key f024092912a5fcd37bf049c5ed988e46
I20260924 23:14:39.070225 30166 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9fc85a526304f33b0d01e7f15387396" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42983 } }
I20260924 23:14:39.070384 30166 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:39.070405 30166 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9fc85a526304f33b0d01e7f15387396, State: Initialized, Role: FOLLOWER
I20260924 23:14:39.070520 30166 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9fc85a526304f33b0d01e7f15387396" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42983 } }
I20260924 23:14:39.070590 30166 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:14:39.070617 30166 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:14:39.070667 30166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:39.071774 30166 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9fc85a526304f33b0d01e7f15387396" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42983 } }
I20260924 23:14:39.071869 30166 leader_election.cc:302] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9fc85a526304f33b0d01e7f15387396; no voters:
I20260924 23:14:39.072021 30166 leader_election.cc:288] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:14:39.072048 30171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:39.072175 30171 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396 [term 1 LEADER]: Becoming Leader. State: Replica: b9fc85a526304f33b0d01e7f15387396, State: Running, Role: LEADER
I20260924 23:14:39.072319 30166 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:14:39.072319 30171 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9fc85a526304f33b0d01e7f15387396" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42983 } }
I20260924 23:14:39.072759 30172 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9fc85a526304f33b0d01e7f15387396" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9fc85a526304f33b0d01e7f15387396" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42983 } } }
I20260924 23:14:39.072840 30172 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:39.073341 30173 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9fc85a526304f33b0d01e7f15387396. Latest consensus state: current_term: 1 leader_uuid: "b9fc85a526304f33b0d01e7f15387396" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9fc85a526304f33b0d01e7f15387396" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42983 } } }
I20260924 23:14:39.073498 30173 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:39.073678 30178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:14:39.074259 30178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:14:39.075982 30178 catalog_manager.cc:1347] Generated new cluster ID: a42805e75e0d463dad6330f2243f700b
I20260924 23:14:39.076035 30178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 23:14:39 dist-test-slave-m0z6 krb5kdc[30085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:14:39.079860 30170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.070090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60544 (local address 127.12.212.62:42983)
0924 23:14:39.070295 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:39.070303 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:39.070329 (+ 26us) server_negotiation.cc:407] Connection header received
0924 23:14:39.070443 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:39.070450 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:39.070507 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:39.070651 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:39.071432 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.072632 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.073689 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.074197 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.074300 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.076736 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:39.076740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:39.076741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:39.076753 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:39.078303 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:39.078636 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:39.078639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:39.078640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:39.078820 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:39.079004 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:39.079005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:39.079006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:39.079268 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:39.079414 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:39.079520 (+ 106us) server_negotiation.cc:299] Negotiation successful
0924 23:14:39.079674 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":108,"threads_started":1}
I20260924 23:14:39.092904 30178 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:14:39.093519 30178 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:14:39.101752 30178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396: Generated new TSK 0
I20260924 23:14:39.101991 30178 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
I20260924 23:14:39.134052 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42983
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:39.214679 30192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:39.214952 30192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:39.214978 30192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:39.216207 30192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:39.216259 30192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:39.216286 30192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:14:39.217810 30192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:39.218065 30192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:39.218297 30192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:39.223071 30197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:39.223066 30198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:39.223064 30200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:39.223322 30192 server_base.cc:1034] running on GCE node
I20260924 23:14:39.223567 30192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:39.223816 30192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:39.224952 30192 hybrid_clock.cc:648] HybridClock initialized: now 1790291679224940 us; error 28 us; skew 500 ppm
Sep 24 23:14:39 dist-test-slave-m0z6 krb5kdc[30085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291679, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:39.227366 30192 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:14:39.227777 30192 webserver.cc:460] Webserver started at http://127.12.212.1:36479/ using document root <none> and password file <none>
I20260924 23:14:39.227981 30192 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:39.228024 30192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:39.228147 30192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:39.229197 30192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "4f689a64701b406ba78e75f6ab13719c"
format_stamp: "Formatted at 2026-09-24 23:14:39 on dist-test-slave-m0z6"
server_key: "1bcf76f64ca5b9df2d1a6c1542e5ef9c"
server_key_iv: "73a3bb271e049963d047fc828a63c921"
server_key_version: "encryptionkey@0"
I20260924 23:14:39.229527 30192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "4f689a64701b406ba78e75f6ab13719c"
format_stamp: "Formatted at 2026-09-24 23:14:39 on dist-test-slave-m0z6"
server_key: "1bcf76f64ca5b9df2d1a6c1542e5ef9c"
server_key_iv: "73a3bb271e049963d047fc828a63c921"
server_key_version: "encryptionkey@0"
I20260924 23:14:39.230930 30192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:14:39.231825 30207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:39.232018 30192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:39.232102 30192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "4f689a64701b406ba78e75f6ab13719c"
format_stamp: "Formatted at 2026-09-24 23:14:39 on dist-test-slave-m0z6"
server_key: "1bcf76f64ca5b9df2d1a6c1542e5ef9c"
server_key_iv: "73a3bb271e049963d047fc828a63c921"
server_key_version: "encryptionkey@0"
I20260924 23:14:39.232167 30192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:39.266834 30192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:39.269366 30192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:39.269498 30192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:39.269703 30192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:39.269970 30192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:39.270001 30192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:39.270023 30192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:39.270040 30192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:39.275733 30192 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:36143
I20260924 23:14:39.275787 30320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:36143 every 8 connection(s)
I20260924 23:14:39.276098 30192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:14:39.279379 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 30192
I20260924 23:14:39.279495 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:14:39.279700 13136 external_mini_cluster.cc:1428] Setting key 31e55cdc668f93f50730463f68cfc5b6
Sep 24 23:14:39 dist-test-slave-m0z6 krb5kdc[30085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291679, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:14:39.286182 30170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.276816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:51521 (local address 127.12.212.62:42983)
0924 23:14:39.276898 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:39.276900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:39.277056 (+ 156us) server_negotiation.cc:407] Connection header received
0924 23:14:39.277533 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:39.277534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:39.277547 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:39.277597 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:39.278561 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.279102 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.280381 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.280585 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.280637 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.283111 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:39.283113 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:39.283113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:39.283121 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:39.284572 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:39.285123 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:39.285124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:39.285124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:39.285182 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:39.285477 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:39.285478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:39.285479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:39.285596 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:39.285648 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:39.285999 (+ 351us) server_negotiation.cc:299] Negotiation successful
0924 23:14:39.286116 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 23:14:39.286415 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.276750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42983 (local address 127.12.212.1:51521)
0924 23:14:39.276920 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:39.276966 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:39.277453 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:39.277656 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:39.277660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:39.278283 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:39.278484 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:39.278489 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.279163 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.279164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:39.280308 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:39.280311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.280660 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.280661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:39.280806 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.281371 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:14:39.281381 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:39.282989 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:39.284665 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:39.284673 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:39.284676 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:39.284994 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:39.285248 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:39.285249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:39.285249 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:39.285408 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:39.285655 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:39.285661 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:39.285848 (+ 187us) client_negotiation.cc:769] Sending connection context
0924 23:14:39.285967 (+ 119us) client_negotiation.cc:240] Negotiation successful
0924 23:14:39.286173 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":66,"threads_started":1}
I20260924 23:14:39.287060 30321 heartbeater.cc:347] Connected to a master server at 127.12.212.62:42983
I20260924 23:14:39.287173 30321 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:39.287362 30321 heartbeater.cc:511] Master 127.12.212.62:42983 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.12.212.2@KRBTEST.COM; defaulting to no policy
I20260924 23:14:39.287962 30130 ts_manager.cc:194] Registered new tserver with Master: 4f689a64701b406ba78e75f6ab13719c (127.12.212.1:36143)
I20260924 23:14:39.288582 30130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:51521
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
I20260924 23:14:39.305594 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42983
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:39.385932 30326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:39.386154 30326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:39.386186 30326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:39.387399 30326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:39.387446 30326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:39.387481 30326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:14:39.388967 30326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:39.389245 30326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:39.389464 30326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:39.394455 30332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:39.394455 30334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:39.394477 30331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:39.394634 30326 server_base.cc:1034] running on GCE node
I20260924 23:14:39.394949 30326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:39.395205 30326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:39.396370 30326 hybrid_clock.cc:648] HybridClock initialized: now 1790291679396355 us; error 31 us; skew 500 ppm
Sep 24 23:14:39 dist-test-slave-m0z6 krb5kdc[30085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291679, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:39.398955 30326 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:14:39.399449 30326 webserver.cc:460] Webserver started at http://127.12.212.2:34701/ using document root <none> and password file <none>
I20260924 23:14:39.399662 30326 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:39.399711 30326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:39.399847 30326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:39.400892 30326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "e3f844ed4eeb42838c7c3a0b081426cd"
format_stamp: "Formatted at 2026-09-24 23:14:39 on dist-test-slave-m0z6"
server_key: "760bbb397ba1cac64a88477277e2c478"
server_key_iv: "1bac199fe7c46299b4b7b7b4adf143c1"
server_key_version: "encryptionkey@0"
I20260924 23:14:39.401240 30326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "e3f844ed4eeb42838c7c3a0b081426cd"
format_stamp: "Formatted at 2026-09-24 23:14:39 on dist-test-slave-m0z6"
server_key: "760bbb397ba1cac64a88477277e2c478"
server_key_iv: "1bac199fe7c46299b4b7b7b4adf143c1"
server_key_version: "encryptionkey@0"
I20260924 23:14:39.402535 30326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:14:39.403518 30341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:39.403797 30326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:39.403891 30326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "e3f844ed4eeb42838c7c3a0b081426cd"
format_stamp: "Formatted at 2026-09-24 23:14:39 on dist-test-slave-m0z6"
server_key: "760bbb397ba1cac64a88477277e2c478"
server_key_iv: "1bac199fe7c46299b4b7b7b4adf143c1"
server_key_version: "encryptionkey@0"
I20260924 23:14:39.403971 30326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:39.420894 30326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:39.423075 30326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:39.423205 30326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:39.423444 30326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:39.423746 30326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:39.423785 30326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:39.423816 30326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:39.423861 30326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:39.429502 30326 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:34079
I20260924 23:14:39.429574 30454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:34079 every 8 connection(s)
I20260924 23:14:39.429860 30326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:14:39.430429 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 30326
I20260924 23:14:39.430536 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:14:39.430774 13136 external_mini_cluster.cc:1428] Setting key 5c219113518be0ec60a26d585dc8ee52
Sep 24 23:14:39 dist-test-slave-m0z6 krb5kdc[30085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291679, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.212.3@KRBTEST.COM; defaulting to no policy
I20260924 23:14:39.439908 30170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.430992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:47135 (local address 127.12.212.62:42983)
0924 23:14:39.431097 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:39.431099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:39.431111 (+ 12us) server_negotiation.cc:407] Connection header received
0924 23:14:39.431493 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:39.431494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:39.431504 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:39.431558 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:39.432799 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.433322 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.434352 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.434556 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.434610 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.437132 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:39.437133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:39.437134 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:39.437141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:39.438450 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:39.438948 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:39.438949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:39.438949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:39.439008 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:39.439279 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:39.439280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:39.439281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:39.439401 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:39.439456 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:39.439724 (+ 268us) server_negotiation.cc:299] Negotiation successful
0924 23:14:39.439815 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260924 23:14:39.440083 30457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.430541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42983 (local address 127.12.212.2:47135)
0924 23:14:39.430754 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:39.430807 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:39.431400 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:39.431645 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:39.431649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:39.432408 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:39.432701 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:39.432708 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.433412 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.433413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:39.434254 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:39.434256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.434627 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.434628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:39.434792 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.435364 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:14:39.435376 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:39.437006 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:39.438539 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:39.438547 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:39.438550 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:39.438877 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:39.439080 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:39.439081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:39.439082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:39.439199 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:39.439455 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:39.439462 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:39.439595 (+ 133us) client_negotiation.cc:769] Sending connection context
0924 23:14:39.439686 (+ 91us) client_negotiation.cc:240] Negotiation successful
0924 23:14:39.439867 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":90,"threads_started":1}
I20260924 23:14:39.440618 30455 heartbeater.cc:347] Connected to a master server at 127.12.212.62:42983
I20260924 23:14:39.440716 30455 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:39.440951 30455 heartbeater.cc:511] Master 127.12.212.62:42983 requested a full tablet report, sending...
I20260924 23:14:39.441450 30130 ts_manager.cc:194] Registered new tserver with Master: e3f844ed4eeb42838c7c3a0b081426cd (127.12.212.2:34079)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.3@KRBTEST.COM" created.
I20260924 23:14:39.441941 30130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:47135
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
I20260924 23:14:39.457132 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42983
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:39.537793 30460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:39.538004 30460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:39.538029 30460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:39.539278 30460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:39.539331 30460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:39.539357 30460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:14:39.540880 30460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:39.541090 30460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:39.541283 30460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:39.546479 30465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:39.546564 30460 server_base.cc:1034] running on GCE node
W20260924 23:14:39.546512 30466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:39.546482 30468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:39.546977 30460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:39.547219 30460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:39.548364 30460 hybrid_clock.cc:648] HybridClock initialized: now 1790291679548352 us; error 30 us; skew 500 ppm
Sep 24 23:14:39 dist-test-slave-m0z6 krb5kdc[30085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291679, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:39.550812 30460 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:14:39.551246 30460 webserver.cc:460] Webserver started at http://127.12.212.3:33501/ using document root <none> and password file <none>
I20260924 23:14:39.551463 30460 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:39.551527 30460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:39.551656 30460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:39.552706 30460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "4b3a563f439a4bb598ca2fcd9fb15eff"
format_stamp: "Formatted at 2026-09-24 23:14:39 on dist-test-slave-m0z6"
server_key: "342e4a0eb56aa381924baabc50fd6f69"
server_key_iv: "0c8f6c5d77cc724fc18bf6fd27144596"
server_key_version: "encryptionkey@0"
I20260924 23:14:39.553035 30460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "4b3a563f439a4bb598ca2fcd9fb15eff"
format_stamp: "Formatted at 2026-09-24 23:14:39 on dist-test-slave-m0z6"
server_key: "342e4a0eb56aa381924baabc50fd6f69"
server_key_iv: "0c8f6c5d77cc724fc18bf6fd27144596"
server_key_version: "encryptionkey@0"
I20260924 23:14:39.554497 30460 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:14:39.555307 30475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:39.555482 30460 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 23:14:39.555572 30460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "4b3a563f439a4bb598ca2fcd9fb15eff"
format_stamp: "Formatted at 2026-09-24 23:14:39 on dist-test-slave-m0z6"
server_key: "342e4a0eb56aa381924baabc50fd6f69"
server_key_iv: "0c8f6c5d77cc724fc18bf6fd27144596"
server_key_version: "encryptionkey@0"
I20260924 23:14:39.555648 30460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:39.567668 30460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:39.569783 30460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:39.569934 30460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:39.570178 30460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:39.570465 30460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:39.570498 30460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:39.570528 30460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:39.570551 30460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:39.576483 30460 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:36233
I20260924 23:14:39.576530 30588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:36233 every 8 connection(s)
I20260924 23:14:39.576841 30460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:14:39.582160 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 30460
I20260924 23:14:39.582283 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:14:39.582437 13136 external_mini_cluster.cc:1428] Setting key 1e0460249f4089abb86180967ad74543
Sep 24 23:14:39 dist-test-slave-m0z6 krb5kdc[30085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291679, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:14:39.586851 30170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.577600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:38275 (local address 127.12.212.62:42983)
0924 23:14:39.577657 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:39.577659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:39.577857 (+ 198us) server_negotiation.cc:407] Connection header received
0924 23:14:39.578367 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:39.578369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:39.578379 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:39.578441 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:39.579580 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.580085 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.581090 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.581334 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.581396 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.584141 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:39.584143 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:39.584143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:39.584150 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:39.585472 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:39.585870 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:39.585871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:39.585872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:39.585925 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:39.586251 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:39.586253 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:39.586253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:39.586378 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:39.586437 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:39.586682 (+ 245us) server_negotiation.cc:299] Negotiation successful
0924 23:14:39.586782 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 23:14:39.587044 30591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.577589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42983 (local address 127.12.212.3:38275)
0924 23:14:39.577734 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:39.577776 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:39.578249 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:39.578504 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:39.578508 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:39.579156 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:39.579431 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:39.579438 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.580162 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.580163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:39.581002 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:39.581005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.581438 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.581439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:39.581590 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.582376 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:14:39.582387 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:39.583990 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:39.585557 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:39.585563 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:39.585566 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:39.585802 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:39.586009 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:39.586010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:39.586010 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:39.586131 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:39.586435 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:39.586438 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:39.586561 (+ 123us) client_negotiation.cc:769] Sending connection context
0924 23:14:39.586642 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 23:14:39.586823 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260924 23:14:39.587463 30589 heartbeater.cc:347] Connected to a master server at 127.12.212.62:42983
I20260924 23:14:39.587546 30589 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:39.587738 30589 heartbeater.cc:511] Master 127.12.212.62:42983 requested a full tablet report, sending...
I20260924 23:14:39.588099 30130 ts_manager.cc:194] Registered new tserver with Master: 4b3a563f439a4bb598ca2fcd9fb15eff (127.12.212.3:36233)
I20260924 23:14:39.588459 30130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:38275
I20260924 23:14:39.594013 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 23:14:39.600914 30170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.595327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60554 (local address 127.12.212.62:42983)
0924 23:14:39.595380 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:39.595382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:39.595470 (+ 88us) server_negotiation.cc:407] Connection header received
0924 23:14:39.595534 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:39.595535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:39.595545 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:39.595593 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:39.596127 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.596632 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.597329 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.597514 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.597565 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.598636 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:39.598637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:39.598637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:39.598645 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:39.599894 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:39.600177 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:39.600178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:39.600178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:39.600254 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:39.600416 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:39.600417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:39.600417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:39.600561 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:39.600646 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:39.600751 (+ 105us) server_negotiation.cc:299] Negotiation successful
0924 23:14:39.600824 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 23:14:39.602420 30130 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:14:39.603971 30130 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:14:39.610316 30600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.605915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:36233 (local address 127.0.0.1:39080)
0924 23:14:39.606186 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:39.606226 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:39.606284 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:39.606578 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:39.606580 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:39.606582 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:39.606645 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:39.606648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.608022 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.608023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:39.609124 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:39.609126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.610001 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.610002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:39.610082 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.610096 (+ 14us) client_negotiation.cc:769] Sending connection context
0924 23:14:39.610173 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 23:14:39.610231 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":47,"thread_start_us":49,"threads_started":1}
I20260924 23:14:39.610415 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.606104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:36143 (local address 127.0.0.1:50462)
0924 23:14:39.606975 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:39.606984 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:39.607028 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:39.607235 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:39.607236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:39.607238 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:39.607289 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:39.607291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.608230 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.608231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:39.609429 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:39.609432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.610167 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.610168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:39.610246 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.610251 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 23:14:39.610298 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 23:14:39.610342 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":847,"mutex_wait_us":64,"thread_start_us":52,"threads_started":1}
I20260924 23:14:39.610499 30601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.605919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:34079 (local address 127.0.0.1:59756)
0924 23:14:39.606223 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:39.606232 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:39.606282 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:39.606927 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:39.606928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:39.606929 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:39.606967 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:39.606968 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.608181 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.608182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:39.609444 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:39.609446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.610284 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.610285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:39.610341 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.610345 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:14:39.610395 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 23:14:39.610437 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":4,"thread_start_us":71,"threads_started":1}
I20260924 23:14:39.610638 30605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.606254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50462 (local address 127.12.212.1:36143)
0924 23:14:39.606419 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:39.606423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:39.607078 (+ 655us) server_negotiation.cc:407] Connection header received
0924 23:14:39.607114 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:39.607116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:39.607127 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:39.607185 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:39.607348 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.608125 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.609527 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.610084 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.610156 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.610215 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:39.610513 (+ 298us) server_negotiation.cc:299] Negotiation successful
0924 23:14:39.610564 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
I20260924 23:14:39.610680 30602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.606006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39080 (local address 127.12.212.3:36233)
0924 23:14:39.606445 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:39.606448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:39.606456 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:14:39.606486 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:39.606487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:39.606500 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:39.606563 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:39.606782 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.607934 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.609213 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.609906 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.609979 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.610039 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:39.610549 (+ 510us) server_negotiation.cc:299] Negotiation successful
0924 23:14:39.610606 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":84,"threads_started":1}
I20260924 23:14:39.610826 30603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.606027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59756 (local address 127.12.212.2:34079)
0924 23:14:39.606734 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:39.606738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:39.606747 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:14:39.606787 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:39.606789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:39.606805 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:39.606875 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:39.607028 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.608088 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.609514 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.610216 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.610282 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.610336 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:39.610678 (+ 342us) server_negotiation.cc:299] Negotiation successful
0924 23:14:39.610752 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":67,"threads_started":1}
I20260924 23:14:39.611377 30523 tablet_service.cc:1467] Processing CreateTablet for tablet 72ea60b8772e4fbfb278f1ede73276bc (DEFAULT_TABLE table=test-table [id=8ca14f69c37749e19085f2dd9709b3da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:39.611377 30255 tablet_service.cc:1467] Processing CreateTablet for tablet 72ea60b8772e4fbfb278f1ede73276bc (DEFAULT_TABLE table=test-table [id=8ca14f69c37749e19085f2dd9709b3da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:39.611480 30389 tablet_service.cc:1467] Processing CreateTablet for tablet 72ea60b8772e4fbfb278f1ede73276bc (DEFAULT_TABLE table=test-table [id=8ca14f69c37749e19085f2dd9709b3da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:39.611740 30523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72ea60b8772e4fbfb278f1ede73276bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:39.611742 30255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72ea60b8772e4fbfb278f1ede73276bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:39.611778 30389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72ea60b8772e4fbfb278f1ede73276bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:39.614403 30606 tablet_bootstrap.cc:492] T 72ea60b8772e4fbfb278f1ede73276bc P e3f844ed4eeb42838c7c3a0b081426cd: Bootstrap starting.
I20260924 23:14:39.615026 30607 tablet_bootstrap.cc:492] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff: Bootstrap starting.
I20260924 23:14:39.615173 30606 tablet_bootstrap.cc:654] T 72ea60b8772e4fbfb278f1ede73276bc P e3f844ed4eeb42838c7c3a0b081426cd: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:39.615684 30607 tablet_bootstrap.cc:654] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:39.615715 30606 log.cc:824] T 72ea60b8772e4fbfb278f1ede73276bc P e3f844ed4eeb42838c7c3a0b081426cd: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:39.616236 30607 log.cc:824] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:39.616473 30606 tablet_bootstrap.cc:492] T 72ea60b8772e4fbfb278f1ede73276bc P e3f844ed4eeb42838c7c3a0b081426cd: No bootstrap required, opened a new log
I20260924 23:14:39.616546 30606 ts_tablet_manager.cc:1397] T 72ea60b8772e4fbfb278f1ede73276bc P e3f844ed4eeb42838c7c3a0b081426cd: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260924 23:14:39.616570 30608 tablet_bootstrap.cc:492] T 72ea60b8772e4fbfb278f1ede73276bc P 4f689a64701b406ba78e75f6ab13719c: Bootstrap starting.
I20260924 23:14:39.616961 30607 tablet_bootstrap.cc:492] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff: No bootstrap required, opened a new log
I20260924 23:14:39.617028 30607 ts_tablet_manager.cc:1397] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:14:39.618543 30606 raft_consensus.cc:348] T 72ea60b8772e4fbfb278f1ede73276bc P e3f844ed4eeb42838c7c3a0b081426cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3a563f439a4bb598ca2fcd9fb15eff" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36233 } } peers { permanent_uuid: "e3f844ed4eeb42838c7c3a0b081426cd" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34079 } } peers { permanent_uuid: "4f689a64701b406ba78e75f6ab13719c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36143 } }
I20260924 23:14:39.618775 30606 raft_consensus.cc:374] T 72ea60b8772e4fbfb278f1ede73276bc P e3f844ed4eeb42838c7c3a0b081426cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:39.618821 30606 raft_consensus.cc:729] T 72ea60b8772e4fbfb278f1ede73276bc P e3f844ed4eeb42838c7c3a0b081426cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3f844ed4eeb42838c7c3a0b081426cd, State: Initialized, Role: FOLLOWER
I20260924 23:14:39.618940 30606 consensus_queue.cc:260] T 72ea60b8772e4fbfb278f1ede73276bc P e3f844ed4eeb42838c7c3a0b081426cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3a563f439a4bb598ca2fcd9fb15eff" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36233 } } peers { permanent_uuid: "e3f844ed4eeb42838c7c3a0b081426cd" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34079 } } peers { permanent_uuid: "4f689a64701b406ba78e75f6ab13719c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36143 } }
I20260924 23:14:39.618894 30607 raft_consensus.cc:348] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3a563f439a4bb598ca2fcd9fb15eff" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36233 } } peers { permanent_uuid: "e3f844ed4eeb42838c7c3a0b081426cd" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34079 } } peers { permanent_uuid: "4f689a64701b406ba78e75f6ab13719c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36143 } }
I20260924 23:14:39.619093 30607 raft_consensus.cc:374] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:39.619138 30607 raft_consensus.cc:729] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3a563f439a4bb598ca2fcd9fb15eff, State: Initialized, Role: FOLLOWER
I20260924 23:14:39.619227 30606 ts_tablet_manager.cc:1428] T 72ea60b8772e4fbfb278f1ede73276bc P e3f844ed4eeb42838c7c3a0b081426cd: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260924 23:14:39.619251 30607 consensus_queue.cc:260] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3a563f439a4bb598ca2fcd9fb15eff" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36233 } } peers { permanent_uuid: "e3f844ed4eeb42838c7c3a0b081426cd" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34079 } } peers { permanent_uuid: "4f689a64701b406ba78e75f6ab13719c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36143 } }
I20260924 23:14:39.619334 30608 tablet_bootstrap.cc:654] T 72ea60b8772e4fbfb278f1ede73276bc P 4f689a64701b406ba78e75f6ab13719c: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:39.619354 30455 heartbeater.cc:503] Master 127.12.212.62:42983 was elected leader, sending a full tablet report...
I20260924 23:14:39.619511 30589 heartbeater.cc:503] Master 127.12.212.62:42983 was elected leader, sending a full tablet report...
I20260924 23:14:39.619742 30607 ts_tablet_manager.cc:1428] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:14:39.619859 30608 log.cc:824] T 72ea60b8772e4fbfb278f1ede73276bc P 4f689a64701b406ba78e75f6ab13719c: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:39.620702 30608 tablet_bootstrap.cc:492] T 72ea60b8772e4fbfb278f1ede73276bc P 4f689a64701b406ba78e75f6ab13719c: No bootstrap required, opened a new log
I20260924 23:14:39.620765 30608 ts_tablet_manager.cc:1397] T 72ea60b8772e4fbfb278f1ede73276bc P 4f689a64701b406ba78e75f6ab13719c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260924 23:14:39.622282 30608 raft_consensus.cc:348] T 72ea60b8772e4fbfb278f1ede73276bc P 4f689a64701b406ba78e75f6ab13719c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3a563f439a4bb598ca2fcd9fb15eff" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36233 } } peers { permanent_uuid: "e3f844ed4eeb42838c7c3a0b081426cd" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34079 } } peers { permanent_uuid: "4f689a64701b406ba78e75f6ab13719c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36143 } }
I20260924 23:14:39.622421 30608 raft_consensus.cc:374] T 72ea60b8772e4fbfb278f1ede73276bc P 4f689a64701b406ba78e75f6ab13719c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:39.622452 30608 raft_consensus.cc:729] T 72ea60b8772e4fbfb278f1ede73276bc P 4f689a64701b406ba78e75f6ab13719c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f689a64701b406ba78e75f6ab13719c, State: Initialized, Role: FOLLOWER
I20260924 23:14:39.622567 30608 consensus_queue.cc:260] T 72ea60b8772e4fbfb278f1ede73276bc P 4f689a64701b406ba78e75f6ab13719c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3a563f439a4bb598ca2fcd9fb15eff" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36233 } } peers { permanent_uuid: "e3f844ed4eeb42838c7c3a0b081426cd" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34079 } } peers { permanent_uuid: "4f689a64701b406ba78e75f6ab13719c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36143 } }
I20260924 23:14:39.622816 30608 ts_tablet_manager.cc:1428] T 72ea60b8772e4fbfb278f1ede73276bc P 4f689a64701b406ba78e75f6ab13719c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 23:14:39.622836 30321 heartbeater.cc:503] Master 127.12.212.62:42983 was elected leader, sending a full tablet report...
W20260924 23:14:39.680401 30456 tablet.cc:2369] T 72ea60b8772e4fbfb278f1ede73276bc P e3f844ed4eeb42838c7c3a0b081426cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:14:39.727789 30612 raft_consensus.cc:482] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:14:39.727910 30612 raft_consensus.cc:504] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3a563f439a4bb598ca2fcd9fb15eff" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36233 } } peers { permanent_uuid: "e3f844ed4eeb42838c7c3a0b081426cd" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34079 } } peers { permanent_uuid: "4f689a64701b406ba78e75f6ab13719c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36143 } }
I20260924 23:14:39.728262 30612 leader_election.cc:288] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3f844ed4eeb42838c7c3a0b081426cd (127.12.212.2:34079), 4f689a64701b406ba78e75f6ab13719c (127.12.212.1:36143)
I20260924 23:14:39.732084 30605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.728570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:40937 (local address 127.12.212.1:36143)
0924 23:14:39.728616 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:39.728618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:39.728694 (+ 76us) server_negotiation.cc:407] Connection header received
0924 23:14:39.728733 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:39.728734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:39.728742 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:39.728780 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:39.728908 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.729601 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.731091 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.731741 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.731800 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.731807 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:39.731976 (+ 169us) server_negotiation.cc:299] Negotiation successful
0924 23:14:39.732025 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 23:14:39.732075 30591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.728424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:34079 (local address 127.12.212.3:54913)
0924 23:14:39.728463 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:39.728476 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:39.728522 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:39.728677 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:39.728678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:39.728680 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:39.728756 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:39.728758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.729661 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.729662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:39.730982 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:39.730985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.731793 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.731795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:39.731875 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.731879 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:14:39.731933 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 23:14:39.731982 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":111}
I20260924 23:14:39.732075 30603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.728497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:54913 (local address 127.12.212.2:34079)
0924 23:14:39.728542 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:39.728544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:39.728554 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:14:39.728585 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:39.728586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:39.728596 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:39.728685 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:39.728819 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.729568 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.731088 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.731690 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.731773 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.731785 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:39.731957 (+ 172us) server_negotiation.cc:299] Negotiation successful
0924 23:14:39.732004 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 23:14:39.732331 30615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.728440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:36143 (local address 127.12.212.3:40937)
0924 23:14:39.728629 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:39.728637 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:39.728685 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:39.728799 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:39.728799 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:39.728801 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:39.728856 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:39.728857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.729675 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.729676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:39.730982 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:39.730985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.731837 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.731838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:39.731898 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.731902 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:14:39.731958 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 23:14:39.732001 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":1,"thread_start_us":117,"threads_started":1}
I20260924 23:14:39.732560 30409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72ea60b8772e4fbfb278f1ede73276bc" candidate_uuid: "4b3a563f439a4bb598ca2fcd9fb15eff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3f844ed4eeb42838c7c3a0b081426cd" is_pre_election: true
I20260924 23:14:39.732628 30275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72ea60b8772e4fbfb278f1ede73276bc" candidate_uuid: "4b3a563f439a4bb598ca2fcd9fb15eff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f689a64701b406ba78e75f6ab13719c" is_pre_election: true
I20260924 23:14:39.732718 30409 raft_consensus.cc:2454] T 72ea60b8772e4fbfb278f1ede73276bc P e3f844ed4eeb42838c7c3a0b081426cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b3a563f439a4bb598ca2fcd9fb15eff in term 0.
I20260924 23:14:39.732770 30275 raft_consensus.cc:2454] T 72ea60b8772e4fbfb278f1ede73276bc P 4f689a64701b406ba78e75f6ab13719c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b3a563f439a4bb598ca2fcd9fb15eff in term 0.
I20260924 23:14:39.732965 30478 leader_election.cc:302] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3a563f439a4bb598ca2fcd9fb15eff, e3f844ed4eeb42838c7c3a0b081426cd; no voters:
I20260924 23:14:39.733075 30612 raft_consensus.cc:2781] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:14:39.733129 30612 raft_consensus.cc:482] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:14:39.733152 30612 raft_consensus.cc:3037] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:39.733844 30612 raft_consensus.cc:504] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3a563f439a4bb598ca2fcd9fb15eff" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36233 } } peers { permanent_uuid: "e3f844ed4eeb42838c7c3a0b081426cd" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34079 } } peers { permanent_uuid: "4f689a64701b406ba78e75f6ab13719c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36143 } }
I20260924 23:14:39.733976 30612 leader_election.cc:288] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [CANDIDATE]: Term 1 election: Requested vote from peers e3f844ed4eeb42838c7c3a0b081426cd (127.12.212.2:34079), 4f689a64701b406ba78e75f6ab13719c (127.12.212.1:36143)
I20260924 23:14:39.734153 30409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72ea60b8772e4fbfb278f1ede73276bc" candidate_uuid: "4b3a563f439a4bb598ca2fcd9fb15eff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3f844ed4eeb42838c7c3a0b081426cd"
I20260924 23:14:39.734169 30275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72ea60b8772e4fbfb278f1ede73276bc" candidate_uuid: "4b3a563f439a4bb598ca2fcd9fb15eff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f689a64701b406ba78e75f6ab13719c"
I20260924 23:14:39.734248 30275 raft_consensus.cc:3037] T 72ea60b8772e4fbfb278f1ede73276bc P 4f689a64701b406ba78e75f6ab13719c [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:39.734252 30409 raft_consensus.cc:3037] T 72ea60b8772e4fbfb278f1ede73276bc P e3f844ed4eeb42838c7c3a0b081426cd [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:39.735092 30275 raft_consensus.cc:2454] T 72ea60b8772e4fbfb278f1ede73276bc P 4f689a64701b406ba78e75f6ab13719c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3a563f439a4bb598ca2fcd9fb15eff in term 1.
I20260924 23:14:39.735165 30409 raft_consensus.cc:2454] T 72ea60b8772e4fbfb278f1ede73276bc P e3f844ed4eeb42838c7c3a0b081426cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3a563f439a4bb598ca2fcd9fb15eff in term 1.
I20260924 23:14:39.735265 30476 leader_election.cc:302] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3a563f439a4bb598ca2fcd9fb15eff, 4f689a64701b406ba78e75f6ab13719c; no voters:
I20260924 23:14:39.735354 30612 raft_consensus.cc:2781] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:39.735507 30612 raft_consensus.cc:686] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [term 1 LEADER]: Becoming Leader. State: Replica: 4b3a563f439a4bb598ca2fcd9fb15eff, State: Running, Role: LEADER
I20260924 23:14:39.735602 30612 consensus_queue.cc:237] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3a563f439a4bb598ca2fcd9fb15eff" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36233 } } peers { permanent_uuid: "e3f844ed4eeb42838c7c3a0b081426cd" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34079 } } peers { permanent_uuid: "4f689a64701b406ba78e75f6ab13719c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36143 } }
I20260924 23:14:39.736594 30130 catalog_manager.cc:5495] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b3a563f439a4bb598ca2fcd9fb15eff (127.12.212.3). New cstate: current_term: 1 leader_uuid: "4b3a563f439a4bb598ca2fcd9fb15eff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3a563f439a4bb598ca2fcd9fb15eff" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 36233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e3f844ed4eeb42838c7c3a0b081426cd" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f689a64701b406ba78e75f6ab13719c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36143 } health_report { overall_health: UNKNOWN } } }
I20260924 23:14:39.754904 30602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.752554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39090 (local address 127.12.212.3:36233)
0924 23:14:39.752610 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:39.752612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:39.752620 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:14:39.752659 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:39.752659 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:39.752676 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:39.752735 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:14:39.752844 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.753358 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.754188 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.754379 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.754429 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.754542 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:14:39.754650 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:14:39.754734 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:39.754771 (+ 37us) server_negotiation.cc:299] Negotiation successful
0924 23:14:39.754792 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 23:14:39.756735 30409 raft_consensus.cc:1261] T 72ea60b8772e4fbfb278f1ede73276bc P e3f844ed4eeb42838c7c3a0b081426cd [term 1 FOLLOWER]: Refusing update from remote peer 4b3a563f439a4bb598ca2fcd9fb15eff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:14:39.756728 30275 raft_consensus.cc:1261] T 72ea60b8772e4fbfb278f1ede73276bc P 4f689a64701b406ba78e75f6ab13719c [term 1 FOLLOWER]: Refusing update from remote peer 4b3a563f439a4bb598ca2fcd9fb15eff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:14:39.757092 30616 consensus_queue.cc:1035] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3f844ed4eeb42838c7c3a0b081426cd" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:14:39.757230 30612 consensus_queue.cc:1035] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f689a64701b406ba78e75f6ab13719c" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 36143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:14:39.759721 30619 mvcc.cc:205] Tried to move back new op lower bound from 7333034720280932352 to 7333034720197730304. Current Snapshot: MvccSnapshot[applied={T|T < 7333034720280932352}]
I20260924 23:14:39.760227 30620 mvcc.cc:205] Tried to move back new op lower bound from 7333034720280932352 to 7333034720197730304. Current Snapshot: MvccSnapshot[applied={T|T < 7333034720280932352}]
I20260924 23:14:39.760730 30621 mvcc.cc:205] Tried to move back new op lower bound from 7333034720280932352 to 7333034720197730304. Current Snapshot: MvccSnapshot[applied={T|T < 7333034720280932352}]
I20260924 23:14:39.761925 30129 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:14:39.762007 30129 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:14:39.762669 30129 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b9fc85a526304f33b0d01e7f15387396: Sending DeleteTablet for 3 replicas of tablet 72ea60b8772e4fbfb278f1ede73276bc
I20260924 23:14:39.763002 30389 tablet_service.cc:1514] Processing DeleteTablet for tablet 72ea60b8772e4fbfb278f1ede73276bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:39 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:59756
I20260924 23:14:39.763002 30523 tablet_service.cc:1514] Processing DeleteTablet for tablet 72ea60b8772e4fbfb278f1ede73276bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:39 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:39080
I20260924 23:14:39.763031 30255 tablet_service.cc:1514] Processing DeleteTablet for tablet 72ea60b8772e4fbfb278f1ede73276bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:39 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:50462
I20260924 23:14:39.763228 30629 tablet_replica.cc:330] T 72ea60b8772e4fbfb278f1ede73276bc P e3f844ed4eeb42838c7c3a0b081426cd: stopping tablet replica
I20260924 23:14:39.763322 30629 raft_consensus.cc:2229] T 72ea60b8772e4fbfb278f1ede73276bc P e3f844ed4eeb42838c7c3a0b081426cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:14:39.763419 30629 raft_consensus.cc:2258] T 72ea60b8772e4fbfb278f1ede73276bc P e3f844ed4eeb42838c7c3a0b081426cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:14:39.763447 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 30192
I20260924 23:14:39.765293 30631 tablet_replica.cc:330] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff: stopping tablet replica
I20260924 23:14:39.765399 30631 raft_consensus.cc:2229] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:14:39.765491 30631 raft_consensus.cc:2258] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:14:39.765905 30631 ts_tablet_manager.cc:1905] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:14:39.767566 30631 ts_tablet_manager.cc:1918] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:14:39.767647 30631 log.cc:1196] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/72ea60b8772e4fbfb278f1ede73276bc
I20260924 23:14:39.767881 30631 ts_tablet_manager.cc:1939] T 72ea60b8772e4fbfb278f1ede73276bc P 4b3a563f439a4bb598ca2fcd9fb15eff: Deleting consensus metadata
I20260924 23:14:39.768296 30114 catalog_manager.cc:4841] TS 4b3a563f439a4bb598ca2fcd9fb15eff (127.12.212.3:36233): tablet 72ea60b8772e4fbfb278f1ede73276bc (table test-table [id=8ca14f69c37749e19085f2dd9709b3da]) successfully deleted
W20260924 23:14:39.769126 30114 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.212.1:36143 (error 108)
I20260924 23:14:39.769630 30600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.769499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:36143 (local address 127.0.0.1:50470)
0924 23:14:39.769552 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:39.769591 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.1:36143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 23:14:39.769671 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 30326
W20260924 23:14:39.770714 30114 catalog_manager.cc:4584] TS 4f689a64701b406ba78e75f6ab13719c (127.12.212.1:36143): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 72ea60b8772e4fbfb278f1ede73276bc: Network error: Client connection negotiation failed: client connection to 127.12.212.1:36143: connect: Connection refused (error 111)
I20260924 23:14:39.774271 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 30460
I20260924 23:14:39.774410 30600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.774285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:34079 (local address 127.0.0.1:59766)
0924 23:14:39.774346 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:39.774376 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.2:34079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260924 23:14:39.778303 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 30098
2026-09-24T23:14:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1935 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-24T23:14:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:14:39Z Disabled control of system clock
I20260924 23:14:39.804606 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:35327
--webserver_interface=127.12.212.62
--webserver_port=0
--builtin_ntp_servers=127.12.212.20:34007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:35327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 23:14:39.884325 30636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:39.884564 30636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:39.884586 30636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:39.885766 30636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:14:39.885808 30636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:39.885819 30636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:39.885830 30636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:14:39.885841 30636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:14:39.887183 30636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:34007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:35327
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:35327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:39.887398 30636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:39.887683 30636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:14:39.892735 30636 server_base.cc:1034] running on GCE node
W20260924 23:14:39.892695 30644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:39.892729 30642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:39.892719 30641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:39.893105 30636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:39.893385 30636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:39.894538 30636 hybrid_clock.cc:648] HybridClock initialized: now 1790291679894525 us; error 28 us; skew 500 ppm
I20260924 23:14:39.895819 30636 webserver.cc:460] Webserver started at http://127.12.212.62:39029/ using document root <none> and password file <none>
I20260924 23:14:39.896040 30636 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:39.896093 30636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:39.896202 30636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:39.897337 30636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "b6b35c8573674d9bb43418872efc1bf4"
format_stamp: "Formatted at 2026-09-24 23:14:39 on dist-test-slave-m0z6"
server_key: "01aa5f2b99917bbdedfc8a9d74eac431"
server_key_iv: "92347f215cf621391b774ab4caedf252"
server_key_version: "encryptionkey@0"
I20260924 23:14:39.897709 30636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "b6b35c8573674d9bb43418872efc1bf4"
format_stamp: "Formatted at 2026-09-24 23:14:39 on dist-test-slave-m0z6"
server_key: "01aa5f2b99917bbdedfc8a9d74eac431"
server_key_iv: "92347f215cf621391b774ab4caedf252"
server_key_version: "encryptionkey@0"
I20260924 23:14:39.899389 30636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260924 23:14:39.900295 30650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:39.900522 30636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:39.900619 30636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "b6b35c8573674d9bb43418872efc1bf4"
format_stamp: "Formatted at 2026-09-24 23:14:39 on dist-test-slave-m0z6"
server_key: "01aa5f2b99917bbdedfc8a9d74eac431"
server_key_iv: "92347f215cf621391b774ab4caedf252"
server_key_version: "encryptionkey@0"
I20260924 23:14:39.900691 30636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:39.911896 30636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:39.912236 30636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:39.912379 30636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:39.916184 30636 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:35327
I20260924 23:14:39.916224 30702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:35327 every 8 connection(s)
I20260924 23:14:39.916555 30636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:14:39.917474 30703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:39.918952 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 30636
I20260924 23:14:39.919041 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:14:39.919211 13136 external_mini_cluster.cc:1428] Setting key 2b807501b3bb5197c7d6a0b75ec0ee1b
I20260924 23:14:39.919986 30703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4: Bootstrap starting.
I20260924 23:14:39.920578 30703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:39.920894 30703 log.cc:824] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:39.921556 30703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4: No bootstrap required, opened a new log
I20260924 23:14:39.923821 30703 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b35c8573674d9bb43418872efc1bf4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 35327 } }
I20260924 23:14:39.923985 30703 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:39.924027 30703 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6b35c8573674d9bb43418872efc1bf4, State: Initialized, Role: FOLLOWER
I20260924 23:14:39.924193 30703 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b35c8573674d9bb43418872efc1bf4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 35327 } }
I20260924 23:14:39.924324 30703 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:14:39.924356 30703 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:14:39.924424 30703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:39.924796 30706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:39.919735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42276 (local address 127.12.212.62:35327)
0924 23:14:39.919897 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:39.919905 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:39.919929 (+ 24us) server_negotiation.cc:407] Connection header received
0924 23:14:39.920356 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:39.920363 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:39.920428 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:39.920542 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:39.920779 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.921814 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.922935 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:39.923371 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:39.923458 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:39.924111 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:39.924116 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:39.924116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:39.924301 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:39.924335 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:39.924346 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:39.924408 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:39.924521 (+ 113us) server_negotiation.cc:299] Negotiation successful
0924 23:14:39.924605 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260924 23:14:39.925069 30703 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b35c8573674d9bb43418872efc1bf4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 35327 } }
I20260924 23:14:39.925173 30703 leader_election.cc:302] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6b35c8573674d9bb43418872efc1bf4; no voters:
I20260924 23:14:39.925328 30703 leader_election.cc:288] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:14:39.925367 30708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:39.925513 30708 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4 [term 1 LEADER]: Becoming Leader. State: Replica: b6b35c8573674d9bb43418872efc1bf4, State: Running, Role: LEADER
I20260924 23:14:39.925640 30703 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:14:39.925654 30708 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b35c8573674d9bb43418872efc1bf4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 35327 } }
I20260924 23:14:39.925982 30708 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6b35c8573674d9bb43418872efc1bf4. Latest consensus state: current_term: 1 leader_uuid: "b6b35c8573674d9bb43418872efc1bf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b35c8573674d9bb43418872efc1bf4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 35327 } } }
I20260924 23:14:39.926122 30708 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:39.926005 30710 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6b35c8573674d9bb43418872efc1bf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b35c8573674d9bb43418872efc1bf4" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 35327 } } }
I20260924 23:14:39.926342 30710 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:39.926554 30717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:14:39.927034 30717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:14:39.928465 30717 catalog_manager.cc:1347] Generated new cluster ID: 2dc4c3a09efd43fe994d18b606cb8abe
I20260924 23:14:39.928515 30717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:14:39.949292 30717 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:14:39.949852 30717 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:14:39.956559 30717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4: Generated new TSK 0
I20260924 23:14:39.956792 30717 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:14:39.960462 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:35327
--builtin_ntp_servers=127.12.212.20:34007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 23:14:40.047492 30727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:40.047842 30727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:40.047899 30727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:40.049811 30727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:40.049882 30727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:40.049924 30727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:14:40.051913 30727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:34007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:35327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:40.052191 30727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:40.052512 30727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:40.059373 30735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:40.059365 30732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:40.059545 30733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:40.059794 30727 server_base.cc:1034] running on GCE node
I20260924 23:14:40.060051 30727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:40.060348 30727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:40.061520 30727 hybrid_clock.cc:648] HybridClock initialized: now 1790291680061495 us; error 39 us; skew 500 ppm
I20260924 23:14:40.062827 30727 webserver.cc:460] Webserver started at http://127.12.212.1:34641/ using document root <none> and password file <none>
I20260924 23:14:40.063098 30727 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:40.063211 30727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:40.063351 30727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:40.064452 30727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "ae559a4adbcf48738c65bcd2d5062dd5"
format_stamp: "Formatted at 2026-09-24 23:14:40 on dist-test-slave-m0z6"
server_key: "953ba8ab590cad2d9aef0bee9926ec8b"
server_key_iv: "406eaf048ae793eb8643d731ca3a0a13"
server_key_version: "encryptionkey@0"
I20260924 23:14:40.064785 30727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "ae559a4adbcf48738c65bcd2d5062dd5"
format_stamp: "Formatted at 2026-09-24 23:14:40 on dist-test-slave-m0z6"
server_key: "953ba8ab590cad2d9aef0bee9926ec8b"
server_key_iv: "406eaf048ae793eb8643d731ca3a0a13"
server_key_version: "encryptionkey@0"
I20260924 23:14:40.066067 30727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:14:40.066887 30741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:40.067052 30727 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:14:40.067142 30727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "ae559a4adbcf48738c65bcd2d5062dd5"
format_stamp: "Formatted at 2026-09-24 23:14:40 on dist-test-slave-m0z6"
server_key: "953ba8ab590cad2d9aef0bee9926ec8b"
server_key_iv: "406eaf048ae793eb8643d731ca3a0a13"
server_key_version: "encryptionkey@0"
I20260924 23:14:40.067243 30727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:40.085075 30727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:40.085448 30727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:40.085592 30727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:40.085827 30727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:40.086162 30727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:40.086199 30727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:40.086229 30727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:40.086262 30727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:40.092229 30727 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:41747
I20260924 23:14:40.092401 30854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:41747 every 8 connection(s)
I20260924 23:14:40.092648 30727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:14:40.093636 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 30727
I20260924 23:14:40.093755 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:14:40.093967 13136 external_mini_cluster.cc:1428] Setting key bf11828173268707b0c521c4b30cc6a1
I20260924 23:14:40.094908 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:35327
--builtin_ntp_servers=127.12.212.20:34007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 23:14:40.098901 30706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:40.093557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:46631 (local address 127.12.212.62:35327)
0924 23:14:40.093606 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:40.093608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:40.094040 (+ 432us) server_negotiation.cc:407] Connection header received
0924 23:14:40.094653 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:40.094654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:40.094666 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:40.094747 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:40.095529 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.096172 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.097267 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.097549 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.097617 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:40.098564 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:40.098566 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:40.098566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:40.098573 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:40.098592 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:40.098597 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:40.098656 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:40.098766 (+ 110us) server_negotiation.cc:299] Negotiation successful
0924 23:14:40.098808 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 23:14:40.099050 30857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:40.093495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:35327 (local address 127.12.212.1:46631)
0924 23:14:40.093826 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:40.093873 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:40.094513 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:40.094825 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:40.094829 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:40.094839 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:40.095433 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:40.095441 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.096295 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.096296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:40.097177 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:40.097180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.097691 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.097692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:40.097859 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:40.098415 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:14:40.098416 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:40.098449 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:14:40.098475 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:40.098664 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:40.098669 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:40.098672 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:14:40.098760 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 23:14:40.098852 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":101,"threads_started":1}
I20260924 23:14:40.099594 30855 heartbeater.cc:347] Connected to a master server at 127.12.212.62:35327
I20260924 23:14:40.099690 30855 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:40.099879 30855 heartbeater.cc:511] Master 127.12.212.62:35327 requested a full tablet report, sending...
I20260924 23:14:40.100484 30667 ts_manager.cc:194] Registered new tserver with Master: ae559a4adbcf48738c65bcd2d5062dd5 (127.12.212.1:41747)
I20260924 23:14:40.101159 30667 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.212.1:46631
W20260924 23:14:40.177366 30858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:40.177593 30858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:40.177626 30858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:40.178949 30858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:40.178995 30858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:40.179031 30858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:14:40.180469 30858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:34007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:35327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:40.180727 30858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:40.181025 30858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:40.186491 30864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:40.186492 30866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:40.186492 30863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:40.187084 30858 server_base.cc:1034] running on GCE node
I20260924 23:14:40.187386 30858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:40.187745 30858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:40.188946 30858 hybrid_clock.cc:648] HybridClock initialized: now 1790291680188908 us; error 56 us; skew 500 ppm
I20260924 23:14:40.190541 30858 webserver.cc:460] Webserver started at http://127.12.212.2:41729/ using document root <none> and password file <none>
I20260924 23:14:40.190850 30858 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:40.190912 30858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:40.191047 30858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:40.192183 30858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "6898026a175d4ff585c2eb4d2c6666f4"
format_stamp: "Formatted at 2026-09-24 23:14:40 on dist-test-slave-m0z6"
server_key: "3a1bd99f08fa9949ab11fb8abe464275"
server_key_iv: "2d9c5e3d1e2576a8139b891e59d5fea7"
server_key_version: "encryptionkey@0"
I20260924 23:14:40.192548 30858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "6898026a175d4ff585c2eb4d2c6666f4"
format_stamp: "Formatted at 2026-09-24 23:14:40 on dist-test-slave-m0z6"
server_key: "3a1bd99f08fa9949ab11fb8abe464275"
server_key_iv: "2d9c5e3d1e2576a8139b891e59d5fea7"
server_key_version: "encryptionkey@0"
I20260924 23:14:40.193917 30858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:14:40.194849 30872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:40.195037 30858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:40.195122 30858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "6898026a175d4ff585c2eb4d2c6666f4"
format_stamp: "Formatted at 2026-09-24 23:14:40 on dist-test-slave-m0z6"
server_key: "3a1bd99f08fa9949ab11fb8abe464275"
server_key_iv: "2d9c5e3d1e2576a8139b891e59d5fea7"
server_key_version: "encryptionkey@0"
I20260924 23:14:40.195204 30858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:40.212363 30858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:40.212692 30858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:40.212821 30858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:40.213047 30858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:40.213384 30858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:40.213419 30858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:40.213447 30858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:40.213464 30858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:40.219137 30858 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:34147
I20260924 23:14:40.219183 30985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:34147 every 8 connection(s)
I20260924 23:14:40.219534 30858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:14:40.220698 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 30858
I20260924 23:14:40.220789 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:14:40.220968 13136 external_mini_cluster.cc:1428] Setting key 1031f3b522d0b363813bd1a0946c685f
I20260924 23:14:40.221951 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:35327
--builtin_ntp_servers=127.12.212.20:34007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 23:14:40.225142 30706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:40.220298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:35013 (local address 127.12.212.62:35327)
0924 23:14:40.220377 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:40.220379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:40.220613 (+ 234us) server_negotiation.cc:407] Connection header received
0924 23:14:40.221077 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:40.221078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:40.221090 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:40.221140 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:40.221903 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.222453 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.223809 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.224005 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.224069 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:40.224885 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:40.224886 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:40.224887 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:40.224892 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:40.224908 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:40.224912 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:40.224957 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:40.225032 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 23:14:40.225061 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260924 23:14:40.225373 30988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:40.220260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:35327 (local address 127.12.212.2:35013)
0924 23:14:40.220436 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:40.220477 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:40.220983 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:40.221245 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:40.221248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:40.221257 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:40.221823 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:40.221829 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.222534 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.222535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:40.223717 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:40.223721 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.224082 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.224083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:40.224209 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:40.224743 (+ 534us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:14:40.224744 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:40.224775 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:14:40.224801 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:40.224966 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:40.224971 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:40.224974 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:14:40.225038 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 23:14:40.225147 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
I20260924 23:14:40.225802 30986 heartbeater.cc:347] Connected to a master server at 127.12.212.62:35327
I20260924 23:14:40.225940 30986 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:40.226153 30986 heartbeater.cc:511] Master 127.12.212.62:35327 requested a full tablet report, sending...
I20260924 23:14:40.226536 30667 ts_manager.cc:194] Registered new tserver with Master: 6898026a175d4ff585c2eb4d2c6666f4 (127.12.212.2:34147)
I20260924 23:14:40.226979 30667 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.212.2:35013
W20260924 23:14:40.304760 30989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:40.304975 30989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:40.304997 30989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:40.306178 30989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:40.306216 30989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:40.306243 30989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:14:40.307658 30989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:34007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:35327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:40.307868 30989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:40.308126 30989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:40.313055 30997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:40.313037 30994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:40.313041 30995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:40.313238 30989 server_base.cc:1034] running on GCE node
I20260924 23:14:40.313448 30989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:40.313695 30989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:40.314831 30989 hybrid_clock.cc:648] HybridClock initialized: now 1790291680314817 us; error 28 us; skew 500 ppm
I20260924 23:14:40.316175 30989 webserver.cc:460] Webserver started at http://127.12.212.3:46433/ using document root <none> and password file <none>
I20260924 23:14:40.316401 30989 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:40.316443 30989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:40.316545 30989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:40.317588 30989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "08884e814b574665a6aef976089c8bc0"
format_stamp: "Formatted at 2026-09-24 23:14:40 on dist-test-slave-m0z6"
server_key: "64f6f2d2d84b59e0d9fa11d2ef3843ab"
server_key_iv: "dba49d781df46779c36b8afdd18f910a"
server_key_version: "encryptionkey@0"
I20260924 23:14:40.317907 30989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "08884e814b574665a6aef976089c8bc0"
format_stamp: "Formatted at 2026-09-24 23:14:40 on dist-test-slave-m0z6"
server_key: "64f6f2d2d84b59e0d9fa11d2ef3843ab"
server_key_iv: "dba49d781df46779c36b8afdd18f910a"
server_key_version: "encryptionkey@0"
I20260924 23:14:40.319300 30989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:14:40.320217 31003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:40.320416 30989 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:14:40.320511 30989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "08884e814b574665a6aef976089c8bc0"
format_stamp: "Formatted at 2026-09-24 23:14:40 on dist-test-slave-m0z6"
server_key: "64f6f2d2d84b59e0d9fa11d2ef3843ab"
server_key_iv: "dba49d781df46779c36b8afdd18f910a"
server_key_version: "encryptionkey@0"
I20260924 23:14:40.320585 30989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:40.338299 30989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:40.338605 30989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:40.338773 30989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:40.339015 30989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:40.339329 30989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:40.339358 30989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:40.339386 30989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:40.339399 30989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:40.344745 30989 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:42261
I20260924 23:14:40.344827 31116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:42261 every 8 connection(s)
I20260924 23:14:40.345112 30989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:14:40.347366 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 30989
I20260924 23:14:40.347478 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:14:40.347687 13136 external_mini_cluster.cc:1428] Setting key 4edcd8f8f26173caf3d03bf8c5126981
I20260924 23:14:40.350767 30706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:40.345909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:54199 (local address 127.12.212.62:35327)
0924 23:14:40.345975 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:40.345978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:40.346245 (+ 267us) server_negotiation.cc:407] Connection header received
0924 23:14:40.346747 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:40.346748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:40.346761 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:40.346826 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:40.347586 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.348203 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.349359 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.349552 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.349607 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:40.350494 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:40.350496 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:40.350496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:40.350501 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:40.350516 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:40.350520 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:40.350562 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:40.350632 (+ 70us) server_negotiation.cc:299] Negotiation successful
0924 23:14:40.350658 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 23:14:40.350994 31119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:40.345893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:35327 (local address 127.12.212.3:54199)
0924 23:14:40.346087 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:40.346129 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:40.346619 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:40.346928 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:40.346931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:40.346941 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:40.347453 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:40.347458 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.348420 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.348422 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:40.349282 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:40.349284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.349619 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.349620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:40.349769 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:40.350312 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:14:40.350313 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:40.350366 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:14:40.350412 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:40.350562 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:40.350564 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:40.350567 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:14:40.350614 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 23:14:40.350763 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260924 23:14:40.351410 31117 heartbeater.cc:347] Connected to a master server at 127.12.212.62:35327
I20260924 23:14:40.351491 31117 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:40.351640 31117 heartbeater.cc:511] Master 127.12.212.62:35327 requested a full tablet report, sending...
I20260924 23:14:40.351944 30667 ts_manager.cc:194] Registered new tserver with Master: 08884e814b574665a6aef976089c8bc0 (127.12.212.3:42261)
I20260924 23:14:40.352274 30667 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.212.3:54199
I20260924 23:14:40.359489 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 23:14:40.363620 30706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:40.360854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42284 (local address 127.12.212.62:35327)
0924 23:14:40.360901 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:40.360903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:40.360936 (+ 33us) server_negotiation.cc:407] Connection header received
0924 23:14:40.361007 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:40.361008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:40.361018 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:40.361059 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:40.361236 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.361743 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.362458 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.362645 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.362714 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:40.363389 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:40.363391 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:40.363391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:40.363396 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:40.363411 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:40.363415 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:40.363460 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:40.363509 (+ 49us) server_negotiation.cc:299] Negotiation successful
0924 23:14:40.363536 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 23:14:40.365067 30667 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42284:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:14:40.366633 30667 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:14:40.372927 31126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:40.368841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:41747 (local address 127.0.0.1:45704)
0924 23:14:40.369244 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:40.369254 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:40.369305 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:40.369570 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:40.369573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:40.369575 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:40.369646 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:40.369648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.370622 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.370623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:40.371763 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:40.371765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.372538 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.372538 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:40.372628 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:40.372645 (+ 17us) client_negotiation.cc:769] Sending connection context
0924 23:14:40.372777 (+ 132us) client_negotiation.cc:240] Negotiation successful
0924 23:14:40.372835 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":2,"thread_start_us":80,"threads_started":1}
I20260924 23:14:40.372954 31130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:40.369174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45704 (local address 127.12.212.1:41747)
0924 23:14:40.369402 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:40.369407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:40.369418 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:14:40.369459 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:40.369461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:40.369475 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:40.369551 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:40.369726 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.370549 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.371841 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.372449 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.372521 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:40.372627 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:40.372819 (+ 192us) server_negotiation.cc:299] Negotiation successful
0924 23:14:40.372863 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":96,"threads_started":1}
I20260924 23:14:40.373231 31125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:40.368841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:34147 (local address 127.0.0.1:40678)
0924 23:14:40.368977 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:40.369037 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:40.369116 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:40.369761 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:40.369762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:40.369764 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:40.369811 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:40.369813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.371184 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.371185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:40.372084 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:40.372086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.373013 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.373014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:40.373084 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:40.373088 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:14:40.373116 (+ 28us) client_negotiation.cc:240] Negotiation successful
0924 23:14:40.373153 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":3,"spinlock_wait_cycles":384,"thread_start_us":83,"threads_started":1}
I20260924 23:14:40.373546 31127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:40.368942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40678 (local address 127.12.212.2:34147)
0924 23:14:40.369429 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:40.369433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:40.369441 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:14:40.369473 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:40.369475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:40.369490 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:40.369879 (+ 389us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:40.369925 (+ 46us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.371101 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.372163 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.372918 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.373299 (+ 381us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:40.373396 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:40.373424 (+ 28us) server_negotiation.cc:299] Negotiation successful
0924 23:14:40.373464 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":116,"threads_started":1}
I20260924 23:14:40.374066 30789 tablet_service.cc:1467] Processing CreateTablet for tablet f533964258124676afc7d42ca81ee2b0 (DEFAULT_TABLE table=test-table [id=88f177ab64584734910ca22d8e05080a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:40.374327 31128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:40.369070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:42261 (local address 127.0.0.1:56922)
0924 23:14:40.369602 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:40.369611 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:40.369653 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:40.369856 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:40.369857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:40.369859 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:40.369895 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:40.369897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.372157 (+ 2260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.372158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:40.373114 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:40.373117 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.374079 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.374080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:40.374134 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:40.374138 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:14:40.374185 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 23:14:40.374252 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":1,"thread_start_us":64,"threads_started":1}
I20260924 23:14:40.374338 31129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:40.369185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56922 (local address 127.12.212.3:42261)
0924 23:14:40.369402 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:40.369407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:40.369665 (+ 258us) server_negotiation.cc:407] Connection header received
0924 23:14:40.369713 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:40.369715 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:40.369730 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:40.369796 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:40.369970 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.371101 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.373195 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.373904 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.373964 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:40.374048 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:40.374223 (+ 175us) server_negotiation.cc:299] Negotiation successful
0924 23:14:40.374262 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":89,"threads_started":1}
I20260924 23:14:40.374264 30920 tablet_service.cc:1467] Processing CreateTablet for tablet f533964258124676afc7d42ca81ee2b0 (DEFAULT_TABLE table=test-table [id=88f177ab64584734910ca22d8e05080a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:40.374485 30789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f533964258124676afc7d42ca81ee2b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:40.374549 30920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f533964258124676afc7d42ca81ee2b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:40.374938 31051 tablet_service.cc:1467] Processing CreateTablet for tablet f533964258124676afc7d42ca81ee2b0 (DEFAULT_TABLE table=test-table [id=88f177ab64584734910ca22d8e05080a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:40.375175 31051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f533964258124676afc7d42ca81ee2b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:40.377607 31131 tablet_bootstrap.cc:492] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0: Bootstrap starting.
I20260924 23:14:40.378243 31131 tablet_bootstrap.cc:654] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:40.378366 31132 tablet_bootstrap.cc:492] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4: Bootstrap starting.
I20260924 23:14:40.378542 31131 log.cc:824] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:40.378835 31133 tablet_bootstrap.cc:492] T f533964258124676afc7d42ca81ee2b0 P ae559a4adbcf48738c65bcd2d5062dd5: Bootstrap starting.
I20260924 23:14:40.379081 31132 tablet_bootstrap.cc:654] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:40.379204 31131 tablet_bootstrap.cc:492] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0: No bootstrap required, opened a new log
I20260924 23:14:40.379272 31131 ts_tablet_manager.cc:1397] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:14:40.379395 31132 log.cc:824] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:40.379508 31133 tablet_bootstrap.cc:654] T f533964258124676afc7d42ca81ee2b0 P ae559a4adbcf48738c65bcd2d5062dd5: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:40.379786 31133 log.cc:824] T f533964258124676afc7d42ca81ee2b0 P ae559a4adbcf48738c65bcd2d5062dd5: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:40.380106 31132 tablet_bootstrap.cc:492] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4: No bootstrap required, opened a new log
I20260924 23:14:40.380167 31132 ts_tablet_manager.cc:1397] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:14:40.380542 31133 tablet_bootstrap.cc:492] T f533964258124676afc7d42ca81ee2b0 P ae559a4adbcf48738c65bcd2d5062dd5: No bootstrap required, opened a new log
I20260924 23:14:40.380609 31133 ts_tablet_manager.cc:1397] T f533964258124676afc7d42ca81ee2b0 P ae559a4adbcf48738c65bcd2d5062dd5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:14:40.381441 31131 raft_consensus.cc:348] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae559a4adbcf48738c65bcd2d5062dd5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41747 } } peers { permanent_uuid: "6898026a175d4ff585c2eb4d2c6666f4" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34147 } } peers { permanent_uuid: "08884e814b574665a6aef976089c8bc0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42261 } }
I20260924 23:14:40.381597 31131 raft_consensus.cc:374] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:40.381629 31131 raft_consensus.cc:729] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08884e814b574665a6aef976089c8bc0, State: Initialized, Role: FOLLOWER
I20260924 23:14:40.381754 31131 consensus_queue.cc:260] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae559a4adbcf48738c65bcd2d5062dd5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41747 } } peers { permanent_uuid: "6898026a175d4ff585c2eb4d2c6666f4" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34147 } } peers { permanent_uuid: "08884e814b574665a6aef976089c8bc0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42261 } }
I20260924 23:14:40.382022 31131 ts_tablet_manager.cc:1428] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:14:40.382051 31117 heartbeater.cc:503] Master 127.12.212.62:35327 was elected leader, sending a full tablet report...
I20260924 23:14:40.382627 31132 raft_consensus.cc:348] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae559a4adbcf48738c65bcd2d5062dd5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41747 } } peers { permanent_uuid: "6898026a175d4ff585c2eb4d2c6666f4" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34147 } } peers { permanent_uuid: "08884e814b574665a6aef976089c8bc0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42261 } }
I20260924 23:14:40.382818 31132 raft_consensus.cc:374] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:40.382860 31132 raft_consensus.cc:729] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6898026a175d4ff585c2eb4d2c6666f4, State: Initialized, Role: FOLLOWER
I20260924 23:14:40.382966 31132 consensus_queue.cc:260] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae559a4adbcf48738c65bcd2d5062dd5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41747 } } peers { permanent_uuid: "6898026a175d4ff585c2eb4d2c6666f4" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34147 } } peers { permanent_uuid: "08884e814b574665a6aef976089c8bc0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42261 } }
I20260924 23:14:40.383212 31132 ts_tablet_manager.cc:1428] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:14:40.383337 30986 heartbeater.cc:503] Master 127.12.212.62:35327 was elected leader, sending a full tablet report...
I20260924 23:14:40.383366 31133 raft_consensus.cc:348] T f533964258124676afc7d42ca81ee2b0 P ae559a4adbcf48738c65bcd2d5062dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae559a4adbcf48738c65bcd2d5062dd5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41747 } } peers { permanent_uuid: "6898026a175d4ff585c2eb4d2c6666f4" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34147 } } peers { permanent_uuid: "08884e814b574665a6aef976089c8bc0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42261 } }
I20260924 23:14:40.383500 31133 raft_consensus.cc:374] T f533964258124676afc7d42ca81ee2b0 P ae559a4adbcf48738c65bcd2d5062dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:40.383522 31133 raft_consensus.cc:729] T f533964258124676afc7d42ca81ee2b0 P ae559a4adbcf48738c65bcd2d5062dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae559a4adbcf48738c65bcd2d5062dd5, State: Initialized, Role: FOLLOWER
I20260924 23:14:40.383627 31133 consensus_queue.cc:260] T f533964258124676afc7d42ca81ee2b0 P ae559a4adbcf48738c65bcd2d5062dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae559a4adbcf48738c65bcd2d5062dd5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41747 } } peers { permanent_uuid: "6898026a175d4ff585c2eb4d2c6666f4" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34147 } } peers { permanent_uuid: "08884e814b574665a6aef976089c8bc0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42261 } }
I20260924 23:14:40.383880 30855 heartbeater.cc:503] Master 127.12.212.62:35327 was elected leader, sending a full tablet report...
I20260924 23:14:40.383872 31133 ts_tablet_manager.cc:1428] T f533964258124676afc7d42ca81ee2b0 P ae559a4adbcf48738c65bcd2d5062dd5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260924 23:14:40.470078 30987 tablet.cc:2369] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:14:40.494352 31137 raft_consensus.cc:482] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:14:40.494483 31137 raft_consensus.cc:504] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae559a4adbcf48738c65bcd2d5062dd5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41747 } } peers { permanent_uuid: "6898026a175d4ff585c2eb4d2c6666f4" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34147 } } peers { permanent_uuid: "08884e814b574665a6aef976089c8bc0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42261 } }
I20260924 23:14:40.494889 31137 leader_election.cc:288] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae559a4adbcf48738c65bcd2d5062dd5 (127.12.212.1:41747), 6898026a175d4ff585c2eb4d2c6666f4 (127.12.212.2:34147)
I20260924 23:14:40.497901 31130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:40.495119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:53605 (local address 127.12.212.1:41747)
0924 23:14:40.495175 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:40.495177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:40.495425 (+ 248us) server_negotiation.cc:407] Connection header received
0924 23:14:40.495472 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:40.495473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:40.495481 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:40.495533 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:40.495692 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.496137 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.497187 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.497578 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.497642 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:40.497652 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:40.497798 (+ 146us) server_negotiation.cc:299] Negotiation successful
0924 23:14:40.497838 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 23:14:40.497908 31127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:40.495201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:33651 (local address 127.12.212.2:34147)
0924 23:14:40.495258 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:40.495259 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:40.495267 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:14:40.495294 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:40.495294 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:40.495302 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:40.495356 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:40.495557 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.496106 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.497146 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.497561 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.497642 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:40.497651 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:40.497801 (+ 150us) server_negotiation.cc:299] Negotiation successful
0924 23:14:40.497842 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 23:14:40.497901 31119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:40.495072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:34147 (local address 127.12.212.3:33651)
0924 23:14:40.495125 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:40.495137 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:40.495189 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:40.495378 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:40.495379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:40.495381 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:40.495453 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:40.495456 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.496248 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.496249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:40.497060 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:40.497063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.497633 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.497634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:40.497724 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:40.497727 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:14:40.497766 (+ 39us) client_negotiation.cc:240] Negotiation successful
0924 23:14:40.497809 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":23}
I20260924 23:14:40.497925 31140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:40.495073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:41747 (local address 127.12.212.3:53605)
0924 23:14:40.495371 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:40.495380 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:40.495426 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:40.495555 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:40.495556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:40.495558 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:40.495627 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:40.495628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.496235 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.496237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:40.497096 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:40.497099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.497653 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.497654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:40.497732 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:40.497736 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:14:40.497787 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 23:14:40.497824 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":79,"thread_start_us":155,"threads_started":1}
I20260924 23:14:40.498296 30809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f533964258124676afc7d42ca81ee2b0" candidate_uuid: "08884e814b574665a6aef976089c8bc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae559a4adbcf48738c65bcd2d5062dd5" is_pre_election: true
I20260924 23:14:40.498298 30940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f533964258124676afc7d42ca81ee2b0" candidate_uuid: "08884e814b574665a6aef976089c8bc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6898026a175d4ff585c2eb4d2c6666f4" is_pre_election: true
I20260924 23:14:40.498461 30809 raft_consensus.cc:2454] T f533964258124676afc7d42ca81ee2b0 P ae559a4adbcf48738c65bcd2d5062dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08884e814b574665a6aef976089c8bc0 in term 0.
I20260924 23:14:40.498461 30940 raft_consensus.cc:2454] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08884e814b574665a6aef976089c8bc0 in term 0.
I20260924 23:14:40.498714 31006 leader_election.cc:302] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08884e814b574665a6aef976089c8bc0, 6898026a175d4ff585c2eb4d2c6666f4; no voters:
I20260924 23:14:40.498869 31137 raft_consensus.cc:2781] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:14:40.498930 31137 raft_consensus.cc:482] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:14:40.498946 31137 raft_consensus.cc:3037] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:40.499641 31137 raft_consensus.cc:504] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae559a4adbcf48738c65bcd2d5062dd5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41747 } } peers { permanent_uuid: "6898026a175d4ff585c2eb4d2c6666f4" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34147 } } peers { permanent_uuid: "08884e814b574665a6aef976089c8bc0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42261 } }
I20260924 23:14:40.499784 31137 leader_election.cc:288] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [CANDIDATE]: Term 1 election: Requested vote from peers ae559a4adbcf48738c65bcd2d5062dd5 (127.12.212.1:41747), 6898026a175d4ff585c2eb4d2c6666f4 (127.12.212.2:34147)
I20260924 23:14:40.499997 30809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f533964258124676afc7d42ca81ee2b0" candidate_uuid: "08884e814b574665a6aef976089c8bc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae559a4adbcf48738c65bcd2d5062dd5"
I20260924 23:14:40.500041 30940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f533964258124676afc7d42ca81ee2b0" candidate_uuid: "08884e814b574665a6aef976089c8bc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6898026a175d4ff585c2eb4d2c6666f4"
I20260924 23:14:40.500087 30809 raft_consensus.cc:3037] T f533964258124676afc7d42ca81ee2b0 P ae559a4adbcf48738c65bcd2d5062dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:40.500114 30940 raft_consensus.cc:3037] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:40.500869 30940 raft_consensus.cc:2454] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08884e814b574665a6aef976089c8bc0 in term 1.
I20260924 23:14:40.500955 30809 raft_consensus.cc:2454] T f533964258124676afc7d42ca81ee2b0 P ae559a4adbcf48738c65bcd2d5062dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08884e814b574665a6aef976089c8bc0 in term 1.
I20260924 23:14:40.501057 31006 leader_election.cc:302] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08884e814b574665a6aef976089c8bc0, 6898026a175d4ff585c2eb4d2c6666f4; no voters:
I20260924 23:14:40.501163 31137 raft_consensus.cc:2781] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:40.501300 31137 raft_consensus.cc:686] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [term 1 LEADER]: Becoming Leader. State: Replica: 08884e814b574665a6aef976089c8bc0, State: Running, Role: LEADER
I20260924 23:14:40.501385 31137 consensus_queue.cc:237] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae559a4adbcf48738c65bcd2d5062dd5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41747 } } peers { permanent_uuid: "6898026a175d4ff585c2eb4d2c6666f4" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34147 } } peers { permanent_uuid: "08884e814b574665a6aef976089c8bc0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42261 } }
I20260924 23:14:40.502200 30667 catalog_manager.cc:5495] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08884e814b574665a6aef976089c8bc0 (127.12.212.3). New cstate: current_term: 1 leader_uuid: "08884e814b574665a6aef976089c8bc0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae559a4adbcf48738c65bcd2d5062dd5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6898026a175d4ff585c2eb4d2c6666f4" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08884e814b574665a6aef976089c8bc0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 42261 } health_report { overall_health: HEALTHY } } }
I20260924 23:14:40.519023 31129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:40.516610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56936 (local address 127.12.212.3:42261)
0924 23:14:40.516692 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:40.516694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:40.516701 (+ 7us) server_negotiation.cc:407] Connection header received
0924 23:14:40.516743 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:40.516744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:40.516762 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:40.516823 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:14:40.516946 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.517421 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.518188 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:40.518428 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:40.518488 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:40.518641 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:14:40.518804 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:14:40.518881 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:40.518899 (+ 18us) server_negotiation.cc:299] Negotiation successful
0924 23:14:40.518917 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 23:14:40.520880 30809 raft_consensus.cc:1261] T f533964258124676afc7d42ca81ee2b0 P ae559a4adbcf48738c65bcd2d5062dd5 [term 1 FOLLOWER]: Refusing update from remote peer 08884e814b574665a6aef976089c8bc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:14:40.520880 30940 raft_consensus.cc:1261] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4 [term 1 FOLLOWER]: Refusing update from remote peer 08884e814b574665a6aef976089c8bc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:14:40.521266 31141 consensus_queue.cc:1035] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae559a4adbcf48738c65bcd2d5062dd5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:14:40.521371 31137 consensus_queue.cc:1035] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6898026a175d4ff585c2eb4d2c6666f4" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 34147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:14:40.524080 31145 mvcc.cc:205] Tried to move back new op lower bound from 7333034723411001344 to 7333034723334402048. Current Snapshot: MvccSnapshot[applied={T|T < 7333034723411001344}]
I20260924 23:14:40.526237 30667 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:14:40.526320 30667 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:14:40.527097 30667 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6b35c8573674d9bb43418872efc1bf4: Sending DeleteTablet for 3 replicas of tablet f533964258124676afc7d42ca81ee2b0
I20260924 23:14:40.527468 31051 tablet_service.cc:1514] Processing DeleteTablet for tablet f533964258124676afc7d42ca81ee2b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:40 UTC) from {username='slave'} at 127.0.0.1:56922
I20260924 23:14:40.527473 30789 tablet_service.cc:1514] Processing DeleteTablet for tablet f533964258124676afc7d42ca81ee2b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:40 UTC) from {username='slave'} at 127.0.0.1:45704
I20260924 23:14:40.527473 30920 tablet_service.cc:1514] Processing DeleteTablet for tablet f533964258124676afc7d42ca81ee2b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:40 UTC) from {username='slave'} at 127.0.0.1:40678
I20260924 23:14:40.527693 31155 tablet_replica.cc:330] T f533964258124676afc7d42ca81ee2b0 P ae559a4adbcf48738c65bcd2d5062dd5: stopping tablet replica
I20260924 23:14:40.527787 31155 raft_consensus.cc:2229] T f533964258124676afc7d42ca81ee2b0 P ae559a4adbcf48738c65bcd2d5062dd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:14:40.527894 31155 raft_consensus.cc:2258] T f533964258124676afc7d42ca81ee2b0 P ae559a4adbcf48738c65bcd2d5062dd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:14:40.527938 31154 tablet_replica.cc:330] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0: stopping tablet replica
I20260924 23:14:40.528023 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 30727
I20260924 23:14:40.528043 31154 raft_consensus.cc:2229] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:14:40.528136 31154 raft_consensus.cc:2258] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:14:40.528263 31156 tablet_replica.cc:330] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4: stopping tablet replica
I20260924 23:14:40.528345 31156 raft_consensus.cc:2229] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:14:40.528422 31156 raft_consensus.cc:2258] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:14:40.528878 31156 ts_tablet_manager.cc:1905] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:14:40.531256 31156 ts_tablet_manager.cc:1918] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:14:40.531340 31156 log.cc:1196] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/f533964258124676afc7d42ca81ee2b0
I20260924 23:14:40.531579 31156 ts_tablet_manager.cc:1939] T f533964258124676afc7d42ca81ee2b0 P 6898026a175d4ff585c2eb4d2c6666f4: Deleting consensus metadata
I20260924 23:14:40.532014 30653 catalog_manager.cc:4841] TS 6898026a175d4ff585c2eb4d2c6666f4 (127.12.212.2:34147): tablet f533964258124676afc7d42ca81ee2b0 (table test-table [id=88f177ab64584734910ca22d8e05080a]) successfully deleted
I20260924 23:14:40.532267 31154 ts_tablet_manager.cc:1905] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 23:14:40.533437 30651 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.212.1:41747 (error 108)
I20260924 23:14:40.533820 31154 ts_tablet_manager.cc:1918] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:14:40.533892 31154 log.cc:1196] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/f533964258124676afc7d42ca81ee2b0
W20260924 23:14:40.533963 31126 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:14:40.533780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:41747 (local address 127.0.0.1:45708)
0924 23:14:40.533828 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:40.533839 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:40.533880 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:40.533918 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.1:41747: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":30}
W20260924 23:14:40.534060 30651 catalog_manager.cc:4584] TS ae559a4adbcf48738c65bcd2d5062dd5 (127.12.212.1:41747): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f533964258124676afc7d42ca81ee2b0: Network error: Client connection negotiation failed: client connection to 127.12.212.1:41747: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260924 23:14:40.534157 31154 ts_tablet_manager.cc:1939] T f533964258124676afc7d42ca81ee2b0 P 08884e814b574665a6aef976089c8bc0: Deleting consensus metadata
I20260924 23:14:40.534340 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 30858
I20260924 23:14:40.534996 30653 catalog_manager.cc:4841] TS 08884e814b574665a6aef976089c8bc0 (127.12.212.3:42261): tablet f533964258124676afc7d42ca81ee2b0 (table test-table [id=88f177ab64584734910ca22d8e05080a]) successfully deleted
I20260924 23:14:40.539132 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 30989
I20260924 23:14:40.543804 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 30636
2026-09-24T23:14:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (764 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:14:40 dist-test-slave-m0z6 krb5kdc[31160](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:14:40 dist-test-slave-m0z6 krb5kdc[31160](info): set up 2 sockets
Sep 24 23:14:40 dist-test-slave-m0z6 krb5kdc[31160](info): commencing operation
krb5kdc: starting...
W20260924 23:14:41.502197 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:14:41 dist-test-slave-m0z6 krb5kdc[31160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:14:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:14:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:14:41.613729 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:41193
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:43741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:41193
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:41.692849 31173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:41.693069 31173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:41.693091 31173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:41.694300 31173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:14:41.694355 31173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:41.694370 31173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:41.694380 31173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:14:41.694391 31173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:14:41.695964 31173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:43741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:41193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:41193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:41.696218 31173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:41.696444 31173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:41.701233 31181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:41.701359 31173 server_base.cc:1034] running on GCE node
W20260924 23:14:41.701197 31179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:41.701278 31178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:41.701689 31173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:41.702008 31173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:41.703197 31173 hybrid_clock.cc:648] HybridClock initialized: now 1790291681703154 us; error 57 us; skew 500 ppm
Sep 24 23:14:41 dist-test-slave-m0z6 krb5kdc[31160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291681, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:41.705699 31173 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:14:41.706121 31173 webserver.cc:460] Webserver started at http://127.12.212.62:39121/ using document root <none> and password file <none>
I20260924 23:14:41.706362 31173 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:41.706403 31173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:41.706497 31173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:41.707744 31173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "981b848fdfdb4bf5aa5ed565f639f3ab"
format_stamp: "Formatted at 2026-09-24 23:14:41 on dist-test-slave-m0z6"
server_key: "2fafe04e9b85f625ed4878444d1b1bba"
server_key_iv: "b09d4149d2de2381ee34bc3220353b5a"
server_key_version: "encryptionkey@0"
I20260924 23:14:41.708071 31173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "981b848fdfdb4bf5aa5ed565f639f3ab"
format_stamp: "Formatted at 2026-09-24 23:14:41 on dist-test-slave-m0z6"
server_key: "2fafe04e9b85f625ed4878444d1b1bba"
server_key_iv: "b09d4149d2de2381ee34bc3220353b5a"
server_key_version: "encryptionkey@0"
I20260924 23:14:41.709436 31173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 23:14:41.710271 31188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:41.710475 31173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:14:41.710567 31173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "981b848fdfdb4bf5aa5ed565f639f3ab"
format_stamp: "Formatted at 2026-09-24 23:14:41 on dist-test-slave-m0z6"
server_key: "2fafe04e9b85f625ed4878444d1b1bba"
server_key_iv: "b09d4149d2de2381ee34bc3220353b5a"
server_key_version: "encryptionkey@0"
I20260924 23:14:41.710633 31173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:41.720503 31173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:41.722838 31173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:41.722962 31173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:41.726847 31173 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:41193
I20260924 23:14:41.726899 31240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:41193 every 8 connection(s)
I20260924 23:14:41.727245 31173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:14:41.728089 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 31173
I20260924 23:14:41.728210 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:14:41.728309 31241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:41.728428 13136 external_mini_cluster.cc:1428] Setting key 0585ca64b1afdc0fc762526e67313190
I20260924 23:14:41.731679 31241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab: Bootstrap starting.
I20260924 23:14:41.732379 31241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:41.732645 31241 log.cc:824] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:41.733364 31241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab: No bootstrap required, opened a new log
Sep 24 23:14:41 dist-test-slave-m0z6 krb5kdc[31160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:14:41.735230 31241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "981b848fdfdb4bf5aa5ed565f639f3ab" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 41193 } }
I20260924 23:14:41.735360 31241 raft_consensus.cc:374] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:41.735392 31241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 981b848fdfdb4bf5aa5ed565f639f3ab, State: Initialized, Role: FOLLOWER
I20260924 23:14:41.735522 31241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "981b848fdfdb4bf5aa5ed565f639f3ab" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 41193 } }
I20260924 23:14:41.735605 31241 raft_consensus.cc:388] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:14:41.735637 31241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:14:41.735692 31241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:41.736276 31241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "981b848fdfdb4bf5aa5ed565f639f3ab" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 41193 } }
I20260924 23:14:41.736387 31241 leader_election.cc:302] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 981b848fdfdb4bf5aa5ed565f639f3ab; no voters:
I20260924 23:14:41.736552 31241 leader_election.cc:288] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:14:41.736594 31246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:41.736742 31246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab [term 1 LEADER]: Becoming Leader. State: Replica: 981b848fdfdb4bf5aa5ed565f639f3ab, State: Running, Role: LEADER
I20260924 23:14:41.736840 31246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "981b848fdfdb4bf5aa5ed565f639f3ab" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 41193 } }
I20260924 23:14:41.736843 31241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:14:41.737197 31246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 981b848fdfdb4bf5aa5ed565f639f3ab. Latest consensus state: current_term: 1 leader_uuid: "981b848fdfdb4bf5aa5ed565f639f3ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "981b848fdfdb4bf5aa5ed565f639f3ab" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 41193 } } }
I20260924 23:14:41.737270 31246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:41.737879 31251 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:14:41.738222 31248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "981b848fdfdb4bf5aa5ed565f639f3ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "981b848fdfdb4bf5aa5ed565f639f3ab" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 41193 } } }
I20260924 23:14:41.738322 31248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:41.738474 31251 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:14:41.739697 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:41.728994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49136 (local address 127.12.212.62:41193)
0924 23:14:41.729234 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:41.729244 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:41.729277 (+ 33us) server_negotiation.cc:407] Connection header received
0924 23:14:41.729816 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:41.729827 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:41.729897 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:41.730054 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:41.730716 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:41.732096 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:41.732919 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:41.733416 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:41.733516 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:41.735836 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:41.735840 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:41.735840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:41.735853 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:41.737813 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:41.738197 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:41.738201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:41.738201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:41.738307 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:41.738543 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:41.738544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:41.738545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:41.738861 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:41.738999 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:41.739351 (+ 352us) server_negotiation.cc:299] Negotiation successful
0924 23:14:41.739505 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":79,"threads_started":1}
I20260924 23:14:41.740694 31251 catalog_manager.cc:1347] Generated new cluster ID: 137a4b250b944e4d9e07299feda05325
I20260924 23:14:41.740738 31251 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:14:41.763484 31251 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:14:41.764086 31251 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:14:41.768446 31251 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab: Generated new TSK 0
I20260924 23:14:41.768630 31251 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
I20260924 23:14:41.797084 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:41193
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:43741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:41.878541 31267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:41.878774 31267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:41.878798 31267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:41.879933 31267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:41.879967 31267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:41.879992 31267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:14:41.881471 31267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:43741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:41193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:41.881669 31267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:41.881853 31267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:41.886504 31275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:41.886504 31273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:41.886536 31272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:41.886731 31267 server_base.cc:1034] running on GCE node
I20260924 23:14:41.886960 31267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:41.887195 31267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:41.888342 31267 hybrid_clock.cc:648] HybridClock initialized: now 1790291681888330 us; error 29 us; skew 500 ppm
Sep 24 23:14:41 dist-test-slave-m0z6 krb5kdc[31160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291681, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:41.890970 31267 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:14:41.891454 31267 webserver.cc:460] Webserver started at http://127.12.212.1:33607/ using document root <none> and password file <none>
I20260924 23:14:41.891683 31267 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:41.891732 31267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:41.891855 31267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:41.892927 31267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "2c0fe0e1f1d24f39bf199a516edd9ce5"
format_stamp: "Formatted at 2026-09-24 23:14:41 on dist-test-slave-m0z6"
server_key: "152067b49abdcd3f86964efa05e8690a"
server_key_iv: "06e7d3f706817dc8c9eece441d192eaf"
server_key_version: "encryptionkey@0"
I20260924 23:14:41.893265 31267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "2c0fe0e1f1d24f39bf199a516edd9ce5"
format_stamp: "Formatted at 2026-09-24 23:14:41 on dist-test-slave-m0z6"
server_key: "152067b49abdcd3f86964efa05e8690a"
server_key_iv: "06e7d3f706817dc8c9eece441d192eaf"
server_key_version: "encryptionkey@0"
I20260924 23:14:41.894550 31267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:14:41.895306 31282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:41.895449 31267 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:14:41.895524 31267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "2c0fe0e1f1d24f39bf199a516edd9ce5"
format_stamp: "Formatted at 2026-09-24 23:14:41 on dist-test-slave-m0z6"
server_key: "152067b49abdcd3f86964efa05e8690a"
server_key_iv: "06e7d3f706817dc8c9eece441d192eaf"
server_key_version: "encryptionkey@0"
I20260924 23:14:41.895608 31267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:41.903529 31267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:41.905386 31267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:41.905529 31267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:41.905758 31267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:41.906046 31267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:41.906080 31267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:41.906109 31267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:41.906133 31267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:41.912226 31267 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:37399
I20260924 23:14:41.912281 31395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:37399 every 8 connection(s)
I20260924 23:14:41.912636 31267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
Sep 24 23:14:41 dist-test-slave-m0z6 krb5kdc[31160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291681, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:14:41.921919 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 31267
I20260924 23:14:41.922030 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:14:41.922242 13136 external_mini_cluster.cc:1428] Setting key 3f0a4d9eb097e715acbc64d02fc24320
I20260924 23:14:41.922281 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:41.913411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:51877 (local address 127.12.212.62:41193)
0924 23:14:41.913489 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:41.913491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:41.913737 (+ 246us) server_negotiation.cc:407] Connection header received
0924 23:14:41.914297 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:41.914298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:41.914312 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:41.914363 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:41.915402 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:41.915946 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:41.917079 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:41.917298 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:41.917356 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:41.919643 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:41.919645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:41.919646 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:41.919654 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:41.920790 (+ 1136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:41.921225 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:41.921226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:41.921227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:41.921286 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:41.921524 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:41.921525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:41.921525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:41.921652 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:41.921707 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:41.922051 (+ 344us) server_negotiation.cc:299] Negotiation successful
0924 23:14:41.922212 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 23:14:41.922403 31398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:41.913384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:41193 (local address 127.12.212.1:51877)
0924 23:14:41.913572 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:41.913654 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:41.914189 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:41.914440 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:41.914444 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:41.915091 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:41.915300 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:41.915306 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:41.916038 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:41.916039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:41.916970 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:41.916973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:41.917375 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:41.917376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:41.917497 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:41.918000 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:14:41.918009 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:41.919535 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:41.920865 (+ 1330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:41.920873 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:41.920875 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:41.921120 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:41.921340 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:41.921341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:41.921341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:41.921474 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:41.921705 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:41.921712 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:41.921877 (+ 165us) client_negotiation.cc:769] Sending connection context
0924 23:14:41.921990 (+ 113us) client_negotiation.cc:240] Negotiation successful
0924 23:14:41.922178 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260924 23:14:41.922829 31396 heartbeater.cc:347] Connected to a master server at 127.12.212.62:41193
I20260924 23:14:41.922920 31396 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:41.923089 31396 heartbeater.cc:511] Master 127.12.212.62:41193 requested a full tablet report, sending...
I20260924 23:14:41.923645 31205 ts_manager.cc:194] Registered new tserver with Master: 2c0fe0e1f1d24f39bf199a516edd9ce5 (127.12.212.1:37399)
I20260924 23:14:41.924420 31205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:51877
WARNING: no policy specified for kudu/127.12.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
I20260924 23:14:41.944541 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:41193
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:43741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:42.024403 31401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:42.024631 31401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:42.024655 31401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:42.025882 31401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:42.025934 31401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:42.025960 31401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:14:42.027494 31401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:43741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:41193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:42.027705 31401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:42.027901 31401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:42.033109 31409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:42.033109 31406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:42.033114 31407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:42.033684 31401 server_base.cc:1034] running on GCE node
I20260924 23:14:42.033900 31401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:42.034133 31401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:42.035286 31401 hybrid_clock.cc:648] HybridClock initialized: now 1790291682035263 us; error 38 us; skew 500 ppm
Sep 24 23:14:42 dist-test-slave-m0z6 krb5kdc[31160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291682, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:42.038398 31401 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:14:42.038921 31401 webserver.cc:460] Webserver started at http://127.12.212.2:41969/ using document root <none> and password file <none>
I20260924 23:14:42.039145 31401 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:42.039189 31401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:42.039312 31401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:42.040365 31401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "006384bf025149a995e0194c6aaf1243"
format_stamp: "Formatted at 2026-09-24 23:14:42 on dist-test-slave-m0z6"
server_key: "91440beceb1728702f4f69d9731d5846"
server_key_iv: "f79bdde8e094cc1e6cf4b30da02f40e7"
server_key_version: "encryptionkey@0"
I20260924 23:14:42.040697 31401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "006384bf025149a995e0194c6aaf1243"
format_stamp: "Formatted at 2026-09-24 23:14:42 on dist-test-slave-m0z6"
server_key: "91440beceb1728702f4f69d9731d5846"
server_key_iv: "f79bdde8e094cc1e6cf4b30da02f40e7"
server_key_version: "encryptionkey@0"
I20260924 23:14:42.041952 31401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:14:42.042726 31416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:42.042892 31401 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:14:42.042985 31401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "006384bf025149a995e0194c6aaf1243"
format_stamp: "Formatted at 2026-09-24 23:14:42 on dist-test-slave-m0z6"
server_key: "91440beceb1728702f4f69d9731d5846"
server_key_iv: "f79bdde8e094cc1e6cf4b30da02f40e7"
server_key_version: "encryptionkey@0"
I20260924 23:14:42.043053 31401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:42.054723 31401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:42.057204 31401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:42.057355 31401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:42.057597 31401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:42.057907 31401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:42.057940 31401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:42.057971 31401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:42.057994 31401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:42.064431 31401 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:39987
I20260924 23:14:42.064474 31529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:39987 every 8 connection(s)
I20260924 23:14:42.064818 31401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:14:42.069128 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 31401
I20260924 23:14:42.069252 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:14:42.069465 13136 external_mini_cluster.cc:1428] Setting key bb6e21c6c13d025a056543f35937726c
Sep 24 23:14:42 dist-test-slave-m0z6 krb5kdc[31160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291682, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:14:42.075793 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.065599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:37759 (local address 127.12.212.62:41193)
0924 23:14:42.065701 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:42.065703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:42.065916 (+ 213us) server_negotiation.cc:407] Connection header received
0924 23:14:42.066410 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:42.066411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:42.066422 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:42.066485 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:42.067476 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.068007 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.069132 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.069471 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.069544 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:42.072450 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:42.072452 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:42.072453 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:42.072461 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:42.074155 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:42.074646 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:42.074647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:42.074648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:42.074763 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:42.075071 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:42.075072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:42.075072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:42.075228 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:42.075304 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:42.075571 (+ 267us) server_negotiation.cc:299] Negotiation successful
0924 23:14:42.075688 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260924 23:14:42.075991 31532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.065577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:41193 (local address 127.12.212.2:37759)
0924 23:14:42.065754 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:42.065800 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:42.066302 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:42.066547 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:42.066550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:42.067181 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:42.067386 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:42.067391 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.068122 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.068124 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:42.069025 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:42.069029 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.069594 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.069596 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:42.069770 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:42.070342 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:14:42.070352 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:42.072293 (+ 1941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:42.074273 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:42.074284 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:42.074287 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:42.074556 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:42.074859 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:42.074860 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:42.074861 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:42.075003 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:42.075296 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:42.075302 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:42.075445 (+ 143us) client_negotiation.cc:769] Sending connection context
0924 23:14:42.075535 (+ 90us) client_negotiation.cc:240] Negotiation successful
0924 23:14:42.075708 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260924 23:14:42.076483 31530 heartbeater.cc:347] Connected to a master server at 127.12.212.62:41193
I20260924 23:14:42.076610 31530 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:42.076859 31530 heartbeater.cc:511] Master 127.12.212.62:41193 requested a full tablet report, sending...
I20260924 23:14:42.077286 31205 ts_manager.cc:194] Registered new tserver with Master: 006384bf025149a995e0194c6aaf1243 (127.12.212.2:39987)
I20260924 23:14:42.077675 31205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:37759
WARNING: no policy specified for kudu/127.12.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
I20260924 23:14:42.095822 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:41193
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:43741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:42.178850 31535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:42.179082 31535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:42.179107 31535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:42.180303 31535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:42.180343 31535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:42.180369 31535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:14:42.181861 31535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:43741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:41193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:42.182090 31535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:42.182287 31535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:42.187309 31543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:42.187318 31541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:42.187318 31540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:42.187525 31535 server_base.cc:1034] running on GCE node
I20260924 23:14:42.187819 31535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:42.188059 31535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:42.189247 31535 hybrid_clock.cc:648] HybridClock initialized: now 1790291682189225 us; error 33 us; skew 500 ppm
Sep 24 23:14:42 dist-test-slave-m0z6 krb5kdc[31160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291682, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:42.192167 31535 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:14:42.192641 31535 webserver.cc:460] Webserver started at http://127.12.212.3:33723/ using document root <none> and password file <none>
I20260924 23:14:42.192857 31535 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:42.192905 31535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:42.193032 31535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:42.194132 31535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "5eb6260b3ca84b57a3276cbc4521fb0a"
format_stamp: "Formatted at 2026-09-24 23:14:42 on dist-test-slave-m0z6"
server_key: "6790bc3b9d9659d3222f6668d181a9a3"
server_key_iv: "963a5a2b9f2121561bc9cee85d2010a3"
server_key_version: "encryptionkey@0"
I20260924 23:14:42.194464 31535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "5eb6260b3ca84b57a3276cbc4521fb0a"
format_stamp: "Formatted at 2026-09-24 23:14:42 on dist-test-slave-m0z6"
server_key: "6790bc3b9d9659d3222f6668d181a9a3"
server_key_iv: "963a5a2b9f2121561bc9cee85d2010a3"
server_key_version: "encryptionkey@0"
I20260924 23:14:42.195813 31535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260924 23:14:42.196520 31550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:42.196683 31535 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:14:42.196768 31535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "5eb6260b3ca84b57a3276cbc4521fb0a"
format_stamp: "Formatted at 2026-09-24 23:14:42 on dist-test-slave-m0z6"
server_key: "6790bc3b9d9659d3222f6668d181a9a3"
server_key_iv: "963a5a2b9f2121561bc9cee85d2010a3"
server_key_version: "encryptionkey@0"
I20260924 23:14:42.196834 31535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:42.208419 31535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:42.210492 31535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:42.210630 31535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:42.210886 31535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:42.211218 31535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:42.211254 31535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:42.211285 31535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:42.211302 31535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:42.216907 31535 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:40549
I20260924 23:14:42.216979 31663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:40549 every 8 connection(s)
I20260924 23:14:42.217289 31535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:14:42.220896 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 31535
I20260924 23:14:42.220997 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:14:42.221202 13136 external_mini_cluster.cc:1428] Setting key 4dba9611b7bc73f908054c42fbab8389
Sep 24 23:14:42 dist-test-slave-m0z6 krb5kdc[31160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291682, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:14:42.227115 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.218054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:43427 (local address 127.12.212.62:41193)
0924 23:14:42.218158 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:42.218160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:42.218357 (+ 197us) server_negotiation.cc:407] Connection header received
0924 23:14:42.218971 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:42.218972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:42.218986 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:42.219039 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:42.220025 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.220559 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.221633 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.221834 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.221891 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:42.224161 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:42.224163 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:42.224163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:42.224171 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:42.225542 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:42.225972 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:42.225973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:42.225973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:42.226027 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:42.226308 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:42.226309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:42.226310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:42.226433 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:42.226491 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:42.226872 (+ 381us) server_negotiation.cc:299] Negotiation successful
0924 23:14:42.226959 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 23:14:42.227248 31666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.218015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:41193 (local address 127.12.212.3:43427)
0924 23:14:42.218222 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:42.218267 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:42.218871 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:42.219114 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:42.219118 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:42.219719 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:42.219924 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:42.219931 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.220660 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.220662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:42.221550 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:42.221553 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.221909 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.221910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:42.222038 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:42.222528 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:14:42.222541 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:42.224055 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:42.225642 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:42.225651 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:42.225653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:42.225888 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:42.226129 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:42.226130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:42.226131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:42.226245 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:42.226491 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:42.226499 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:42.226723 (+ 224us) client_negotiation.cc:769] Sending connection context
0924 23:14:42.226841 (+ 118us) client_negotiation.cc:240] Negotiation successful
0924 23:14:42.227015 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":94,"threads_started":1}
I20260924 23:14:42.227738 31664 heartbeater.cc:347] Connected to a master server at 127.12.212.62:41193
I20260924 23:14:42.227829 31664 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:42.228005 31664 heartbeater.cc:511] Master 127.12.212.62:41193 requested a full tablet report, sending...
I20260924 23:14:42.228430 31205 ts_manager.cc:194] Registered new tserver with Master: 5eb6260b3ca84b57a3276cbc4521fb0a (127.12.212.3:40549)
I20260924 23:14:42.228833 31205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:43427
I20260924 23:14:42.232821 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 23:14:42.240379 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.234206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49146 (local address 127.12.212.62:41193)
0924 23:14:42.234310 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:42.234312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:42.234398 (+ 86us) server_negotiation.cc:407] Connection header received
0924 23:14:42.234445 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:42.234445 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:42.234454 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:42.234499 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:42.235104 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.235597 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.236352 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.236579 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.236633 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:42.237909 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:42.237910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:42.237910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:42.237917 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:42.239188 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:42.239574 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:42.239575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:42.239576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:42.239647 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:42.239847 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:42.239847 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:42.239848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:42.240005 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:42.240097 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:42.240231 (+ 134us) server_negotiation.cc:299] Negotiation successful
0924 23:14:42.240289 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260924 23:14:42.241852 31205 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:14:42.243562 31205 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:14:42.249818 31672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.245595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:40549 (local address 127.0.0.1:53114)
0924 23:14:42.245942 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:42.245981 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:42.246040 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:42.246390 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:42.246391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:42.246393 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:42.246452 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:42.246454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.247534 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.247535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:42.248597 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:42.248600 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.249508 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.249509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:42.249594 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:42.249610 (+ 16us) client_negotiation.cc:769] Sending connection context
0924 23:14:42.249676 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 23:14:42.249735 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":57,"threads_started":1}
I20260924 23:14:42.249825 31675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.245785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53114 (local address 127.12.212.3:40549)
0924 23:14:42.246186 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:42.246191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:42.246199 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:14:42.246231 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:42.246233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:42.246246 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:42.246314 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:42.246523 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.247442 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.248690 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.249439 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.249498 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:42.249561 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:42.249696 (+ 135us) server_negotiation.cc:299] Negotiation successful
0924 23:14:42.249737 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":76,"threads_started":1}
I20260924 23:14:42.250198 31676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.245596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:39987 (local address 127.0.0.1:57284)
0924 23:14:42.245943 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:42.245984 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:42.246042 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:42.246321 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:42.246343 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:42.246346 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:42.246431 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:42.246433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.248067 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.248068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:42.248990 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:42.248993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.249851 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.249853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:42.250049 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:42.250055 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 23:14:42.250084 (+ 29us) client_negotiation.cc:240] Negotiation successful
0924 23:14:42.250124 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":38,"thread_start_us":60,"threads_started":1}
I20260924 23:14:42.250372 31673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.245821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:37399 (local address 127.0.0.1:46040)
0924 23:14:42.246142 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:42.246150 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:42.246195 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:42.246725 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:42.246725 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:42.246728 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:42.246814 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:42.246815 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.248067 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.248068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:42.249206 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:42.249208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.250168 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.250169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:42.250229 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:42.250233 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:14:42.250278 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 23:14:42.250315 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":30,"thread_start_us":49,"threads_started":1}
I20260924 23:14:42.250533 31674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.245714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57284 (local address 127.12.212.2:39987)
0924 23:14:42.246155 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:42.246160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:42.246169 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:14:42.246217 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:42.246219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:42.246235 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:42.246314 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:42.246788 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.247949 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.249081 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.249755 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.250326 (+ 571us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:42.250391 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:42.250417 (+ 26us) server_negotiation.cc:299] Negotiation successful
0924 23:14:42.250460 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":122,"threads_started":1}
I20260924 23:14:42.250634 31677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.245921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46040 (local address 127.12.212.1:37399)
0924 23:14:42.246109 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:42.246114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:42.246188 (+ 74us) server_negotiation.cc:407] Connection header received
0924 23:14:42.246549 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:42.246551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:42.246568 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:42.246639 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:42.246895 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.247971 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.249288 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.250024 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.250082 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:42.250137 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:42.250517 (+ 380us) server_negotiation.cc:299] Negotiation successful
0924 23:14:42.250563 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":92,"threads_started":1}
I20260924 23:14:42.251304 31330 tablet_service.cc:1467] Processing CreateTablet for tablet d231153f19264a6ca8134da18f8ee08f (DEFAULT_TABLE table=test-table [id=975528019bb44c0596369cabaf4152da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:42.251426 31464 tablet_service.cc:1467] Processing CreateTablet for tablet d231153f19264a6ca8134da18f8ee08f (DEFAULT_TABLE table=test-table [id=975528019bb44c0596369cabaf4152da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:42.251611 31330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d231153f19264a6ca8134da18f8ee08f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:42.251776 31464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d231153f19264a6ca8134da18f8ee08f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:42.254029 31678 tablet_bootstrap.cc:492] T d231153f19264a6ca8134da18f8ee08f P 2c0fe0e1f1d24f39bf199a516edd9ce5: Bootstrap starting.
I20260924 23:14:42.254314 31598 tablet_service.cc:1467] Processing CreateTablet for tablet d231153f19264a6ca8134da18f8ee08f (DEFAULT_TABLE table=test-table [id=975528019bb44c0596369cabaf4152da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:42.254637 31678 tablet_bootstrap.cc:654] T d231153f19264a6ca8134da18f8ee08f P 2c0fe0e1f1d24f39bf199a516edd9ce5: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:42.254673 31598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d231153f19264a6ca8134da18f8ee08f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:42.255103 31678 log.cc:824] T d231153f19264a6ca8134da18f8ee08f P 2c0fe0e1f1d24f39bf199a516edd9ce5: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:42.255748 31678 tablet_bootstrap.cc:492] T d231153f19264a6ca8134da18f8ee08f P 2c0fe0e1f1d24f39bf199a516edd9ce5: No bootstrap required, opened a new log
I20260924 23:14:42.255803 31678 ts_tablet_manager.cc:1397] T d231153f19264a6ca8134da18f8ee08f P 2c0fe0e1f1d24f39bf199a516edd9ce5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:14:42.257061 31681 tablet_bootstrap.cc:492] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a: Bootstrap starting.
I20260924 23:14:42.257145 31678 raft_consensus.cc:348] T d231153f19264a6ca8134da18f8ee08f P 2c0fe0e1f1d24f39bf199a516edd9ce5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb6260b3ca84b57a3276cbc4521fb0a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40549 } } peers { permanent_uuid: "006384bf025149a995e0194c6aaf1243" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 39987 } } peers { permanent_uuid: "2c0fe0e1f1d24f39bf199a516edd9ce5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37399 } }
I20260924 23:14:42.257324 31678 raft_consensus.cc:374] T d231153f19264a6ca8134da18f8ee08f P 2c0fe0e1f1d24f39bf199a516edd9ce5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:42.257366 31678 raft_consensus.cc:729] T d231153f19264a6ca8134da18f8ee08f P 2c0fe0e1f1d24f39bf199a516edd9ce5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0fe0e1f1d24f39bf199a516edd9ce5, State: Initialized, Role: FOLLOWER
I20260924 23:14:42.257416 31679 tablet_bootstrap.cc:492] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243: Bootstrap starting.
I20260924 23:14:42.257500 31678 consensus_queue.cc:260] T d231153f19264a6ca8134da18f8ee08f P 2c0fe0e1f1d24f39bf199a516edd9ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb6260b3ca84b57a3276cbc4521fb0a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40549 } } peers { permanent_uuid: "006384bf025149a995e0194c6aaf1243" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 39987 } } peers { permanent_uuid: "2c0fe0e1f1d24f39bf199a516edd9ce5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37399 } }
I20260924 23:14:42.257704 31681 tablet_bootstrap.cc:654] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:42.257773 31678 ts_tablet_manager.cc:1428] T d231153f19264a6ca8134da18f8ee08f P 2c0fe0e1f1d24f39bf199a516edd9ce5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:14:42.257890 31396 heartbeater.cc:503] Master 127.12.212.62:41193 was elected leader, sending a full tablet report...
I20260924 23:14:42.258044 31679 tablet_bootstrap.cc:654] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:42.258208 31681 log.cc:824] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:42.258543 31679 log.cc:824] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:42.259068 31681 tablet_bootstrap.cc:492] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a: No bootstrap required, opened a new log
I20260924 23:14:42.259141 31681 ts_tablet_manager.cc:1397] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:14:42.259331 31679 tablet_bootstrap.cc:492] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243: No bootstrap required, opened a new log
I20260924 23:14:42.259392 31679 ts_tablet_manager.cc:1397] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 23:14:42.260522 31681 raft_consensus.cc:348] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb6260b3ca84b57a3276cbc4521fb0a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40549 } } peers { permanent_uuid: "006384bf025149a995e0194c6aaf1243" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 39987 } } peers { permanent_uuid: "2c0fe0e1f1d24f39bf199a516edd9ce5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37399 } }
I20260924 23:14:42.260677 31681 raft_consensus.cc:374] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:42.260700 31681 raft_consensus.cc:729] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eb6260b3ca84b57a3276cbc4521fb0a, State: Initialized, Role: FOLLOWER
I20260924 23:14:42.260797 31681 consensus_queue.cc:260] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb6260b3ca84b57a3276cbc4521fb0a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40549 } } peers { permanent_uuid: "006384bf025149a995e0194c6aaf1243" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 39987 } } peers { permanent_uuid: "2c0fe0e1f1d24f39bf199a516edd9ce5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37399 } }
I20260924 23:14:42.260764 31679 raft_consensus.cc:348] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb6260b3ca84b57a3276cbc4521fb0a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40549 } } peers { permanent_uuid: "006384bf025149a995e0194c6aaf1243" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 39987 } } peers { permanent_uuid: "2c0fe0e1f1d24f39bf199a516edd9ce5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37399 } }
I20260924 23:14:42.260934 31679 raft_consensus.cc:374] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:42.260980 31679 raft_consensus.cc:729] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 006384bf025149a995e0194c6aaf1243, State: Initialized, Role: FOLLOWER
I20260924 23:14:42.261076 31664 heartbeater.cc:503] Master 127.12.212.62:41193 was elected leader, sending a full tablet report...
I20260924 23:14:42.261061 31681 ts_tablet_manager.cc:1428] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:14:42.261099 31679 consensus_queue.cc:260] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb6260b3ca84b57a3276cbc4521fb0a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40549 } } peers { permanent_uuid: "006384bf025149a995e0194c6aaf1243" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 39987 } } peers { permanent_uuid: "2c0fe0e1f1d24f39bf199a516edd9ce5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37399 } }
I20260924 23:14:42.261385 31679 ts_tablet_manager.cc:1428] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 23:14:42.261405 31530 heartbeater.cc:503] Master 127.12.212.62:41193 was elected leader, sending a full tablet report...
W20260924 23:14:42.315405 31531 tablet.cc:2369] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 23:14:42.413415 31397 tablet.cc:2369] T d231153f19264a6ca8134da18f8ee08f P 2c0fe0e1f1d24f39bf199a516edd9ce5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 23:14:42.467875 31665 tablet.cc:2369] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:14:42.513291 31686 raft_consensus.cc:482] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:14:42.513443 31686 raft_consensus.cc:504] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb6260b3ca84b57a3276cbc4521fb0a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40549 } } peers { permanent_uuid: "006384bf025149a995e0194c6aaf1243" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 39987 } } peers { permanent_uuid: "2c0fe0e1f1d24f39bf199a516edd9ce5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37399 } }
I20260924 23:14:42.513937 31686 leader_election.cc:288] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5eb6260b3ca84b57a3276cbc4521fb0a (127.12.212.3:40549), 2c0fe0e1f1d24f39bf199a516edd9ce5 (127.12.212.1:37399)
I20260924 23:14:42.516877 31675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.514256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:48673 (local address 127.12.212.3:40549)
0924 23:14:42.514314 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:42.514316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:42.514326 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:14:42.514358 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:42.514359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:42.514370 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:42.514430 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:42.514637 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.515189 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.516151 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.516554 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.516618 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:42.516628 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:42.516772 (+ 144us) server_negotiation.cc:299] Negotiation successful
0924 23:14:42.516807 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 23:14:42.516877 31532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.514125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:40549 (local address 127.12.212.2:48673)
0924 23:14:42.514193 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:42.514206 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:42.514257 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:42.514472 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:42.514473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:42.514475 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:42.514558 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:42.514560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.515276 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.515277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:42.516057 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:42.516059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.516633 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.516634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:42.516708 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:42.516711 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:14:42.516755 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 23:14:42.516800 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260924 23:14:42.516947 31677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.514213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:33883 (local address 127.12.212.1:37399)
0924 23:14:42.514271 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:42.514274 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:42.514471 (+ 197us) server_negotiation.cc:407] Connection header received
0924 23:14:42.514500 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:42.514501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:42.514511 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:42.514564 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:42.514781 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.515299 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.516268 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.516637 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.516699 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:42.516708 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:42.516846 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 23:14:42.516884 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 23:14:42.517001 31687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.514137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:37399 (local address 127.12.212.2:33883)
0924 23:14:42.514331 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:42.514349 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:42.514419 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:42.514588 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:42.514589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:42.514590 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:42.514663 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:42.514664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.515378 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.515379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:42.516168 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:42.516170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.516718 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.516719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:42.516782 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:42.516786 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:14:42.516831 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 23:14:42.516869 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":107,"threads_started":1}
I20260924 23:14:42.517361 31618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d231153f19264a6ca8134da18f8ee08f" candidate_uuid: "006384bf025149a995e0194c6aaf1243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5eb6260b3ca84b57a3276cbc4521fb0a" is_pre_election: true
I20260924 23:14:42.517357 31350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d231153f19264a6ca8134da18f8ee08f" candidate_uuid: "006384bf025149a995e0194c6aaf1243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0fe0e1f1d24f39bf199a516edd9ce5" is_pre_election: true
I20260924 23:14:42.517531 31350 raft_consensus.cc:2454] T d231153f19264a6ca8134da18f8ee08f P 2c0fe0e1f1d24f39bf199a516edd9ce5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 006384bf025149a995e0194c6aaf1243 in term 0.
I20260924 23:14:42.517531 31618 raft_consensus.cc:2454] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 006384bf025149a995e0194c6aaf1243 in term 0.
I20260924 23:14:42.517772 31417 leader_election.cc:302] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 006384bf025149a995e0194c6aaf1243, 2c0fe0e1f1d24f39bf199a516edd9ce5; no voters:
I20260924 23:14:42.517954 31686 raft_consensus.cc:2781] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:14:42.517994 31686 raft_consensus.cc:482] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:14:42.518015 31686 raft_consensus.cc:3037] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:42.518805 31686 raft_consensus.cc:504] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb6260b3ca84b57a3276cbc4521fb0a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40549 } } peers { permanent_uuid: "006384bf025149a995e0194c6aaf1243" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 39987 } } peers { permanent_uuid: "2c0fe0e1f1d24f39bf199a516edd9ce5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37399 } }
I20260924 23:14:42.518953 31686 leader_election.cc:288] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [CANDIDATE]: Term 1 election: Requested vote from peers 5eb6260b3ca84b57a3276cbc4521fb0a (127.12.212.3:40549), 2c0fe0e1f1d24f39bf199a516edd9ce5 (127.12.212.1:37399)
I20260924 23:14:42.519129 31350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d231153f19264a6ca8134da18f8ee08f" candidate_uuid: "006384bf025149a995e0194c6aaf1243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0fe0e1f1d24f39bf199a516edd9ce5"
I20260924 23:14:42.519148 31618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d231153f19264a6ca8134da18f8ee08f" candidate_uuid: "006384bf025149a995e0194c6aaf1243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5eb6260b3ca84b57a3276cbc4521fb0a"
I20260924 23:14:42.519225 31618 raft_consensus.cc:3037] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:42.519225 31350 raft_consensus.cc:3037] T d231153f19264a6ca8134da18f8ee08f P 2c0fe0e1f1d24f39bf199a516edd9ce5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:42.520102 31618 raft_consensus.cc:2454] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 006384bf025149a995e0194c6aaf1243 in term 1.
I20260924 23:14:42.520102 31350 raft_consensus.cc:2454] T d231153f19264a6ca8134da18f8ee08f P 2c0fe0e1f1d24f39bf199a516edd9ce5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 006384bf025149a995e0194c6aaf1243 in term 1.
I20260924 23:14:42.520310 31417 leader_election.cc:302] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 006384bf025149a995e0194c6aaf1243, 2c0fe0e1f1d24f39bf199a516edd9ce5; no voters:
I20260924 23:14:42.520412 31686 raft_consensus.cc:2781] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:42.520552 31686 raft_consensus.cc:686] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [term 1 LEADER]: Becoming Leader. State: Replica: 006384bf025149a995e0194c6aaf1243, State: Running, Role: LEADER
I20260924 23:14:42.520642 31686 consensus_queue.cc:237] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb6260b3ca84b57a3276cbc4521fb0a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40549 } } peers { permanent_uuid: "006384bf025149a995e0194c6aaf1243" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 39987 } } peers { permanent_uuid: "2c0fe0e1f1d24f39bf199a516edd9ce5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37399 } }
I20260924 23:14:42.521562 31204 catalog_manager.cc:5495] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 reported cstate change: term changed from 0 to 1, leader changed from <none> to 006384bf025149a995e0194c6aaf1243 (127.12.212.2). New cstate: current_term: 1 leader_uuid: "006384bf025149a995e0194c6aaf1243" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb6260b3ca84b57a3276cbc4521fb0a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "006384bf025149a995e0194c6aaf1243" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 39987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c0fe0e1f1d24f39bf199a516edd9ce5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37399 } health_report { overall_health: UNKNOWN } } }
I20260924 23:14:42.535159 31674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.532571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57298 (local address 127.12.212.2:39987)
0924 23:14:42.532618 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:42.532620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:42.532629 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:14:42.532733 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:42.532733 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:42.532753 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:42.532814 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:14:42.532922 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.533422 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.534337 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.534544 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.534612 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:42.534822 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:14:42.534917 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:14:42.534994 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:42.535029 (+ 35us) server_negotiation.cc:299] Negotiation successful
0924 23:14:42.535041 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 23:14:42.537256 31350 raft_consensus.cc:1261] T d231153f19264a6ca8134da18f8ee08f P 2c0fe0e1f1d24f39bf199a516edd9ce5 [term 1 FOLLOWER]: Refusing update from remote peer 006384bf025149a995e0194c6aaf1243: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:14:42.537256 31618 raft_consensus.cc:1261] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a [term 1 FOLLOWER]: Refusing update from remote peer 006384bf025149a995e0194c6aaf1243: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:14:42.537672 31686 consensus_queue.cc:1035] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0fe0e1f1d24f39bf199a516edd9ce5" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 37399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:14:42.537889 31688 consensus_queue.cc:1035] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5eb6260b3ca84b57a3276cbc4521fb0a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:14:42.540786 31692 mvcc.cc:205] Tried to move back new op lower bound from 7333034731669094400 to 7333034731605045248. Current Snapshot: MvccSnapshot[applied={T|T < 7333034731669094400}]
I20260924 23:14:42.542820 31204 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:14:42.542934 31204 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:14:42.543696 31204 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 981b848fdfdb4bf5aa5ed565f639f3ab: Sending DeleteTablet for 3 replicas of tablet d231153f19264a6ca8134da18f8ee08f
I20260924 23:14:42.544054 31464 tablet_service.cc:1514] Processing DeleteTablet for tablet d231153f19264a6ca8134da18f8ee08f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:42 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:57284
I20260924 23:14:42.544072 31598 tablet_service.cc:1514] Processing DeleteTablet for tablet d231153f19264a6ca8134da18f8ee08f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:42 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:53114
I20260924 23:14:42.544397 31330 tablet_service.cc:1514] Processing DeleteTablet for tablet d231153f19264a6ca8134da18f8ee08f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:42 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:46040
I20260924 23:14:42.544550 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 31267
I20260924 23:14:42.545915 31702 tablet_replica.cc:330] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a: stopping tablet replica
I20260924 23:14:42.546036 31702 raft_consensus.cc:2229] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:14:42.546147 31702 raft_consensus.cc:2258] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:14:42.546620 31702 ts_tablet_manager.cc:1905] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:14:42.547888 31701 tablet_replica.cc:330] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243: stopping tablet replica
I20260924 23:14:42.548038 31701 raft_consensus.cc:2229] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:14:42.548149 31701 raft_consensus.cc:2258] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:14:42.548331 31702 ts_tablet_manager.cc:1918] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:14:42.548393 31702 log.cc:1196] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/d231153f19264a6ca8134da18f8ee08f
I20260924 23:14:42.548620 31702 ts_tablet_manager.cc:1939] T d231153f19264a6ca8134da18f8ee08f P 5eb6260b3ca84b57a3276cbc4521fb0a: Deleting consensus metadata
I20260924 23:14:42.549064 31192 catalog_manager.cc:4841] TS 5eb6260b3ca84b57a3276cbc4521fb0a (127.12.212.3:40549): tablet d231153f19264a6ca8134da18f8ee08f (table test-table [id=975528019bb44c0596369cabaf4152da]) successfully deleted
I20260924 23:14:42.549356 31701 ts_tablet_manager.cc:1905] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:14:42.550727 31701 ts_tablet_manager.cc:1918] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:14:42.550802 31701 log.cc:1196] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/d231153f19264a6ca8134da18f8ee08f
I20260924 23:14:42.551054 31701 ts_tablet_manager.cc:1939] T d231153f19264a6ca8134da18f8ee08f P 006384bf025149a995e0194c6aaf1243: Deleting consensus metadata
W20260924 23:14:42.551034 31189 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.212.1:37399 (error 108)
I20260924 23:14:42.551535 31673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.551416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:37399 (local address 127.0.0.1:46042)
0924 23:14:42.551458 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:42.551498 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.1:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260924 23:14:42.551560 31191 catalog_manager.cc:4841] TS 006384bf025149a995e0194c6aaf1243 (127.12.212.2:39987): tablet d231153f19264a6ca8134da18f8ee08f (table test-table [id=975528019bb44c0596369cabaf4152da]) successfully deleted
I20260924 23:14:42.551630 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 31401
W20260924 23:14:42.552881 31189 catalog_manager.cc:4584] TS 2c0fe0e1f1d24f39bf199a516edd9ce5 (127.12.212.1:37399): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d231153f19264a6ca8134da18f8ee08f: Network error: Client connection negotiation failed: client connection to 127.12.212.1:37399: connect: Connection refused (error 111)
I20260924 23:14:42.556756 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 31535
I20260924 23:14:42.561820 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 31173
2026-09-24T23:14:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2018 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-24T23:14:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:14:42Z Disabled control of system clock
I20260924 23:14:42.588653 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:44451
--webserver_interface=127.12.212.62
--webserver_port=0
--builtin_ntp_servers=127.12.212.20:36865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:44451
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 23:14:42.670187 31708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:42.670418 31708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:42.670439 31708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:42.671651 31708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:14:42.671702 31708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:42.671713 31708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:42.671728 31708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:14:42.671741 31708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:14:42.673182 31708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:36865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:44451
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:44451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:42.673408 31708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:42.673666 31708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:42.678619 31716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:42.678536 31714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:42.678582 31708 server_base.cc:1034] running on GCE node
W20260924 23:14:42.678539 31713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:42.679028 31708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:42.679440 31708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:42.680702 31708 hybrid_clock.cc:648] HybridClock initialized: now 1790291682680678 us; error 46 us; skew 500 ppm
I20260924 23:14:42.686239 31708 webserver.cc:460] Webserver started at http://127.12.212.62:42321/ using document root <none> and password file <none>
I20260924 23:14:42.686580 31708 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:42.686651 31708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:42.686841 31708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:42.688043 31708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "68838c6d8238472fb5b81e5865361ae7"
format_stamp: "Formatted at 2026-09-24 23:14:42 on dist-test-slave-m0z6"
server_key: "95ea509352a579a901c012d5db05ff33"
server_key_iv: "b533e4146520eefe0c550defaddb8511"
server_key_version: "encryptionkey@0"
I20260924 23:14:42.688447 31708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "68838c6d8238472fb5b81e5865361ae7"
format_stamp: "Formatted at 2026-09-24 23:14:42 on dist-test-slave-m0z6"
server_key: "95ea509352a579a901c012d5db05ff33"
server_key_iv: "b533e4146520eefe0c550defaddb8511"
server_key_version: "encryptionkey@0"
I20260924 23:14:42.689859 31708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:14:42.690837 31722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:42.691097 31708 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:42.691197 31708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "68838c6d8238472fb5b81e5865361ae7"
format_stamp: "Formatted at 2026-09-24 23:14:42 on dist-test-slave-m0z6"
server_key: "95ea509352a579a901c012d5db05ff33"
server_key_iv: "b533e4146520eefe0c550defaddb8511"
server_key_version: "encryptionkey@0"
I20260924 23:14:42.691293 31708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:42.703161 31708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:42.703471 31708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:42.703600 31708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:42.707512 31708 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:44451
I20260924 23:14:42.707558 31774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:44451 every 8 connection(s)
I20260924 23:14:42.707914 31708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:14:42.708897 31775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:42.711371 31775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7: Bootstrap starting.
I20260924 23:14:42.711978 31775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:42.712262 31775 log.cc:824] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:42.713040 31775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7: No bootstrap required, opened a new log
I20260924 23:14:42.713423 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 31708
I20260924 23:14:42.713541 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:14:42.713730 13136 external_mini_cluster.cc:1428] Setting key bfc07ab9788f53832bea38fff12fd519
I20260924 23:14:42.715312 31775 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68838c6d8238472fb5b81e5865361ae7" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 44451 } }
I20260924 23:14:42.715526 31775 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:42.715574 31775 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68838c6d8238472fb5b81e5865361ae7, State: Initialized, Role: FOLLOWER
I20260924 23:14:42.715720 31775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68838c6d8238472fb5b81e5865361ae7" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 44451 } }
I20260924 23:14:42.715796 31775 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:14:42.715827 31775 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:14:42.715875 31775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:42.716485 31775 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68838c6d8238472fb5b81e5865361ae7" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 44451 } }
I20260924 23:14:42.716578 31775 leader_election.cc:302] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68838c6d8238472fb5b81e5865361ae7; no voters:
I20260924 23:14:42.716765 31775 leader_election.cc:288] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:14:42.716814 31780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:42.716970 31780 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7 [term 1 LEADER]: Becoming Leader. State: Replica: 68838c6d8238472fb5b81e5865361ae7, State: Running, Role: LEADER
I20260924 23:14:42.717032 31775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:14:42.717056 31780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68838c6d8238472fb5b81e5865361ae7" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 44451 } }
I20260924 23:14:42.717478 31780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68838c6d8238472fb5b81e5865361ae7. Latest consensus state: current_term: 1 leader_uuid: "68838c6d8238472fb5b81e5865361ae7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68838c6d8238472fb5b81e5865361ae7" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 44451 } } }
I20260924 23:14:42.717548 31780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:42.718129 31786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:14:42.718580 31786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:14:42.718920 31782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68838c6d8238472fb5b81e5865361ae7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68838c6d8238472fb5b81e5865361ae7" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 44451 } } }
I20260924 23:14:42.719672 31782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:42.720155 31786 catalog_manager.cc:1347] Generated new cluster ID: dcf0519d89064ce7949647185312df08
I20260924 23:14:42.720206 31786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:14:42.720604 31779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.714257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41214 (local address 127.12.212.62:44451)
0924 23:14:42.714476 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:42.714485 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:42.714511 (+ 26us) server_negotiation.cc:407] Connection header received
0924 23:14:42.715195 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:42.715203 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:42.715279 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:42.715415 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:42.715708 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.717102 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.718291 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.718961 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.719086 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:42.719898 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:42.719901 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:42.719902 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:42.720117 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:42.720159 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:42.720172 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:42.720223 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:42.720324 (+ 101us) server_negotiation.cc:299] Negotiation successful
0924 23:14:42.720405 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":96,"threads_started":1}
I20260924 23:14:42.737418 31786 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:14:42.738006 31786 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:14:42.743491 31786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7: Generated new TSK 0
I20260924 23:14:42.743703 31786 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:14:42.748000 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:44451
--builtin_ntp_servers=127.12.212.20:36865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 23:14:42.831869 31799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:42.832089 31799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:42.832113 31799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:42.833395 31799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:42.833441 31799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:42.833469 31799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:14:42.834913 31799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:36865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:44451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:42.835179 31799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:42.835492 31799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:42.841416 31804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:42.841620 31805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:42.841631 31807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:42.842190 31799 server_base.cc:1034] running on GCE node
I20260924 23:14:42.842442 31799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:42.842760 31799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:42.843943 31799 hybrid_clock.cc:648] HybridClock initialized: now 1790291682843920 us; error 33 us; skew 500 ppm
I20260924 23:14:42.845568 31799 webserver.cc:460] Webserver started at http://127.12.212.1:38831/ using document root <none> and password file <none>
I20260924 23:14:42.845863 31799 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:42.845922 31799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:42.846057 31799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:42.847548 31799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "bd59cdf3e31b4946a2bb39883ada8797"
format_stamp: "Formatted at 2026-09-24 23:14:42 on dist-test-slave-m0z6"
server_key: "600b259a9fe082a26b12729f36a37cae"
server_key_iv: "7fdb2144280c26906f4fb71558f3e788"
server_key_version: "encryptionkey@0"
I20260924 23:14:42.848055 31799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "bd59cdf3e31b4946a2bb39883ada8797"
format_stamp: "Formatted at 2026-09-24 23:14:42 on dist-test-slave-m0z6"
server_key: "600b259a9fe082a26b12729f36a37cae"
server_key_iv: "7fdb2144280c26906f4fb71558f3e788"
server_key_version: "encryptionkey@0"
I20260924 23:14:42.849843 31799 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260924 23:14:42.850791 31813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:42.850979 31799 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:42.851079 31799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "bd59cdf3e31b4946a2bb39883ada8797"
format_stamp: "Formatted at 2026-09-24 23:14:42 on dist-test-slave-m0z6"
server_key: "600b259a9fe082a26b12729f36a37cae"
server_key_iv: "7fdb2144280c26906f4fb71558f3e788"
server_key_version: "encryptionkey@0"
I20260924 23:14:42.851176 31799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:42.866155 31799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:42.866496 31799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:42.866613 31799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:42.866912 31799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:42.867228 31799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:42.867267 31799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:42.867290 31799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:42.867301 31799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:42.872885 31799 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:41273
I20260924 23:14:42.872939 31926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:41273 every 8 connection(s)
I20260924 23:14:42.873281 31799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:14:42.878396 31779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.873995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:60209 (local address 127.12.212.62:44451)
0924 23:14:42.874061 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:42.874063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:42.874284 (+ 221us) server_negotiation.cc:407] Connection header received
0924 23:14:42.874866 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:42.874869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:42.874881 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:42.874933 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:42.875612 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.876120 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.877096 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.877308 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.877361 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:42.878131 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:42.878133 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:42.878133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:42.878138 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:42.878152 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:42.878156 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:42.878208 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:42.878282 (+ 74us) server_negotiation.cc:299] Negotiation successful
0924 23:14:42.878314 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 23:14:42.878599 31929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:42.873969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:44451 (local address 127.12.212.1:60209)
0924 23:14:42.874160 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:42.874198 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:42.874757 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:42.875004 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:42.875008 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:42.875017 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:42.875519 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:42.875524 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.876215 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.876216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:42.877010 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:42.877013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:42.877378 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:42.877379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:42.877505 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:42.877998 (+ 493us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:14:42.877999 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:42.878028 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:14:42.878050 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:42.878210 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:42.878215 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:42.878218 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:14:42.878282 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 23:14:42.878386 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260924 23:14:42.879127 31927 heartbeater.cc:347] Connected to a master server at 127.12.212.62:44451
I20260924 23:14:42.879221 31927 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:42.879396 31927 heartbeater.cc:511] Master 127.12.212.62:44451 requested a full tablet report, sending...
I20260924 23:14:42.879907 31739 ts_manager.cc:194] Registered new tserver with Master: bd59cdf3e31b4946a2bb39883ada8797 (127.12.212.1:41273)
I20260924 23:14:42.880489 31739 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.212.1:60209
I20260924 23:14:42.883425 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 31799
I20260924 23:14:42.883525 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:14:42.883697 13136 external_mini_cluster.cc:1428] Setting key 4a210fb0b5caa888413858b51c895684
I20260924 23:14:42.884753 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:44451
--builtin_ntp_servers=127.12.212.20:36865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 23:14:42.966282 31930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:42.966509 31930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:42.966531 31930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:42.967796 31930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:42.967850 31930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:42.967876 31930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:14:42.969419 31930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:36865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:44451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:42.969656 31930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:42.969942 31930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:42.975100 31935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:42.975169 31938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:42.975175 31936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:42.975214 31930 server_base.cc:1034] running on GCE node
I20260924 23:14:42.975579 31930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:42.975871 31930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:42.977058 31930 hybrid_clock.cc:648] HybridClock initialized: now 1790291682977033 us; error 37 us; skew 500 ppm
I20260924 23:14:42.978507 31930 webserver.cc:460] Webserver started at http://127.12.212.2:43843/ using document root <none> and password file <none>
I20260924 23:14:42.978828 31930 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:42.978891 31930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:42.979041 31930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:42.980168 31930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "ccc036ee4e2c409481f108bae0290abc"
format_stamp: "Formatted at 2026-09-24 23:14:42 on dist-test-slave-m0z6"
server_key: "96596bf8894fdcf8afe52b3be81e47c8"
server_key_iv: "4fceab4afdc06865649f3c0345a752da"
server_key_version: "encryptionkey@0"
I20260924 23:14:42.980507 31930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "ccc036ee4e2c409481f108bae0290abc"
format_stamp: "Formatted at 2026-09-24 23:14:42 on dist-test-slave-m0z6"
server_key: "96596bf8894fdcf8afe52b3be81e47c8"
server_key_iv: "4fceab4afdc06865649f3c0345a752da"
server_key_version: "encryptionkey@0"
I20260924 23:14:42.981808 31930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:14:42.982645 31944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:42.982818 31930 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:14:42.982893 31930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "ccc036ee4e2c409481f108bae0290abc"
format_stamp: "Formatted at 2026-09-24 23:14:42 on dist-test-slave-m0z6"
server_key: "96596bf8894fdcf8afe52b3be81e47c8"
server_key_iv: "4fceab4afdc06865649f3c0345a752da"
server_key_version: "encryptionkey@0"
I20260924 23:14:42.982959 31930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:42.997198 31930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:42.997539 31930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:42.997675 31930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:42.997907 31930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:42.998253 31930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:42.998288 31930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:42.998319 31930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:42.998337 31930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:43.003840 31930 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:44525
I20260924 23:14:43.003906 32057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:44525 every 8 connection(s)
I20260924 23:14:43.004243 31930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:14:43.009765 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 31930
I20260924 23:14:43.009889 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:14:43.010015 31779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:43.005123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:49681 (local address 127.12.212.62:44451)
0924 23:14:43.005223 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:43.005225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:43.005405 (+ 180us) server_negotiation.cc:407] Connection header received
0924 23:14:43.005931 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:43.005933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:43.005945 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:43.006010 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:43.006823 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.007356 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.008517 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.008747 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.008815 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:43.009745 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:43.009747 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:43.009747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:43.009753 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:43.009769 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:43.009772 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:43.009815 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:43.009901 (+ 86us) server_negotiation.cc:299] Negotiation successful
0924 23:14:43.009930 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260924 23:14:43.010078 13136 external_mini_cluster.cc:1428] Setting key bc7341d2a365f6d285cf0111c2346de2
I20260924 23:14:43.010306 32060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:43.005071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:44451 (local address 127.12.212.2:49681)
0924 23:14:43.005262 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:43.005308 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:43.005815 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:43.006084 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:43.006088 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:43.006097 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:43.006678 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:43.006683 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.007477 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.007478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:43.008407 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:43.008411 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.008860 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.008861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:43.008996 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:43.009563 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:14:43.009564 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:43.009598 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:14:43.009630 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:43.009825 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:43.009831 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:43.009834 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:14:43.009899 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 23:14:43.010026 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260924 23:14:43.010785 32058 heartbeater.cc:347] Connected to a master server at 127.12.212.62:44451
I20260924 23:14:43.010890 32058 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:43.011077 32058 heartbeater.cc:511] Master 127.12.212.62:44451 requested a full tablet report, sending...
I20260924 23:14:43.011077 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:44451
--builtin_ntp_servers=127.12.212.20:36865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 23:14:43.011459 31739 ts_manager.cc:194] Registered new tserver with Master: ccc036ee4e2c409481f108bae0290abc (127.12.212.2:44525)
I20260924 23:14:43.011902 31739 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.212.2:49681
W20260924 23:14:43.093149 32061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:43.093391 32061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:43.093422 32061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:43.094750 32061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:43.094801 32061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:43.094837 32061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:14:43.096333 32061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:36865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:44451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:43.096611 32061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:43.096925 32061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:43.102065 32067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:43.102083 32066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:43.102115 32061 server_base.cc:1034] running on GCE node
W20260924 23:14:43.102059 32069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:43.102485 32061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:43.102806 32061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:43.103969 32061 hybrid_clock.cc:648] HybridClock initialized: now 1790291683103953 us; error 37 us; skew 500 ppm
I20260924 23:14:43.105273 32061 webserver.cc:460] Webserver started at http://127.12.212.3:36597/ using document root <none> and password file <none>
I20260924 23:14:43.105521 32061 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:43.105572 32061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:43.105686 32061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:43.106907 32061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "0bdda79d65ad454f950585236c8e672d"
format_stamp: "Formatted at 2026-09-24 23:14:43 on dist-test-slave-m0z6"
server_key: "409045099f063ac6f96d245be133b348"
server_key_iv: "2b995ca7b4128b83975c91e1963e92b7"
server_key_version: "encryptionkey@0"
I20260924 23:14:43.107265 32061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "0bdda79d65ad454f950585236c8e672d"
format_stamp: "Formatted at 2026-09-24 23:14:43 on dist-test-slave-m0z6"
server_key: "409045099f063ac6f96d245be133b348"
server_key_iv: "2b995ca7b4128b83975c91e1963e92b7"
server_key_version: "encryptionkey@0"
I20260924 23:14:43.108585 32061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:14:43.109432 32075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:43.109652 32061 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:43.109754 32061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "0bdda79d65ad454f950585236c8e672d"
format_stamp: "Formatted at 2026-09-24 23:14:43 on dist-test-slave-m0z6"
server_key: "409045099f063ac6f96d245be133b348"
server_key_iv: "2b995ca7b4128b83975c91e1963e92b7"
server_key_version: "encryptionkey@0"
I20260924 23:14:43.109824 32061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:43.120638 32061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:43.120975 32061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:43.121120 32061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:43.121351 32061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:43.121685 32061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:43.121721 32061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:43.121748 32061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:43.121765 32061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:43.127476 32061 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:43267
I20260924 23:14:43.127539 32188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:43267 every 8 connection(s)
I20260924 23:14:43.127856 32061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:14:43.133271 31779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:43.128608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:57205 (local address 127.12.212.62:44451)
0924 23:14:43.128687 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:43.128689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:43.128981 (+ 292us) server_negotiation.cc:407] Connection header received
0924 23:14:43.129495 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:43.129496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:43.129506 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:43.129569 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:43.130301 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.130802 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.131914 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.132103 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.132158 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:43.133030 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:43.133031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:43.133031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:43.133036 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:43.133050 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:43.133054 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:43.133092 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:43.133155 (+ 63us) server_negotiation.cc:299] Negotiation successful
0924 23:14:43.133209 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 23:14:43.133445 32191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:43.128568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:44451 (local address 127.12.212.3:57205)
0924 23:14:43.128782 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:43.128839 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:43.129391 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:43.129682 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:43.129686 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:43.129695 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:43.130217 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:43.130222 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.130909 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.130910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:43.131829 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:43.131832 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.132247 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.132248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:43.132373 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:43.132898 (+ 525us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:14:43.132898 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:43.132929 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:14:43.132954 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:43.133101 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:43.133103 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:43.133106 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:14:43.133152 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 23:14:43.133238 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":99,"threads_started":1}
I20260924 23:14:43.133910 32189 heartbeater.cc:347] Connected to a master server at 127.12.212.62:44451
I20260924 23:14:43.133997 32189 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:43.134145 32189 heartbeater.cc:511] Master 127.12.212.62:44451 requested a full tablet report, sending...
I20260924 23:14:43.134495 31739 ts_manager.cc:194] Registered new tserver with Master: 0bdda79d65ad454f950585236c8e672d (127.12.212.3:43267)
I20260924 23:14:43.134910 31739 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.212.3:57205
I20260924 23:14:43.136426 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 32061
I20260924 23:14:43.136512 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:14:43.136670 13136 external_mini_cluster.cc:1428] Setting key 6aba6f23b52c10ecd3470e71cb199962
I20260924 23:14:43.137540 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 23:14:43.141774 31779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:43.138920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41226 (local address 127.12.212.62:44451)
0924 23:14:43.138962 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:43.138964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:43.139074 (+ 110us) server_negotiation.cc:407] Connection header received
0924 23:14:43.139129 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:43.139130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:43.139139 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:43.139189 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:43.139323 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.139839 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.140646 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.140834 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.140883 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:43.141529 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:43.141531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:43.141531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:43.141535 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:43.141549 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:43.141553 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:43.141623 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:43.141700 (+ 77us) server_negotiation.cc:299] Negotiation successful
0924 23:14:43.141713 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 23:14:43.143438 31739 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:14:43.144958 31739 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:14:43.151804 32200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:43.147179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54394 (local address 127.12.212.2:44525)
0924 23:14:43.148042 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:43.148046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:43.148055 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:14:43.148092 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:43.148094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:43.148109 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:43.148197 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:43.148296 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.149264 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.150362 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.151280 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.151372 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:43.151470 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:43.151672 (+ 202us) server_negotiation.cc:299] Negotiation successful
0924 23:14:43.151716 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":810,"thread_start_us":74,"threads_started":1}
I20260924 23:14:43.151795 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:43.147067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:44525 (local address 127.0.0.1:54394)
0924 23:14:43.147254 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:43.147296 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:43.147362 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:43.148195 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:43.148196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:43.148198 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:43.148238 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:43.148240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.149336 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.149338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:43.150269 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:43.150272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.151396 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.151398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:43.151556 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:43.151559 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:14:43.151620 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 23:14:43.151693 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":27,"thread_start_us":117,"threads_started":1}
I20260924 23:14:43.151868 32201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:43.147334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57748 (local address 127.12.212.3:43267)
0924 23:14:43.147528 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:43.147534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:43.147545 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:14:43.147581 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:43.147583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:43.147596 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:43.147666 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:43.147846 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.148977 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.150406 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.151308 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.151372 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:43.151470 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:43.151743 (+ 273us) server_negotiation.cc:299] Negotiation successful
0924 23:14:43.151791 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":86,"threads_started":1}
I20260924 23:14:43.152069 32199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:43.147079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:43267 (local address 127.0.0.1:57748)
0924 23:14:43.147457 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:43.147464 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:43.147497 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:43.147680 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:43.147683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:43.147684 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:43.147773 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:43.147775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.149088 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.149089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:43.150321 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:43.150324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.151397 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.151398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:43.151516 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:43.151534 (+ 18us) client_negotiation.cc:769] Sending connection context
0924 23:14:43.151653 (+ 119us) client_negotiation.cc:240] Negotiation successful
0924 23:14:43.151703 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":49,"thread_start_us":95,"threads_started":1}
I20260924 23:14:43.152667 32123 tablet_service.cc:1467] Processing CreateTablet for tablet c68d8ad86d514389b49455a709887c8e (DEFAULT_TABLE table=test-table [id=4e46a85b320d4e918465a51369fe40bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:43.152667 31992 tablet_service.cc:1467] Processing CreateTablet for tablet c68d8ad86d514389b49455a709887c8e (DEFAULT_TABLE table=test-table [id=4e46a85b320d4e918465a51369fe40bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:43.153023 32198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:43.147067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:41273 (local address 127.0.0.1:46582)
0924 23:14:43.147395 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:43.147404 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:43.147442 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:43.147921 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:43.147922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:43.147923 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:43.147960 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:43.147961 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.148926 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.148927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:43.150322 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:43.150324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.152732 (+ 2408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.152733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:43.152798 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:43.152802 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:14:43.152896 (+ 94us) client_negotiation.cc:240] Negotiation successful
0924 23:14:43.152948 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":38,"thread_start_us":71,"threads_started":1}
I20260924 23:14:43.153044 32123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c68d8ad86d514389b49455a709887c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:43.153044 31992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c68d8ad86d514389b49455a709887c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:43.153193 32202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:43.147393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46582 (local address 127.12.212.1:41273)
0924 23:14:43.147764 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:43.147767 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:43.147774 (+ 7us) server_negotiation.cc:407] Connection header received
0924 23:14:43.147823 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:43.147824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:43.147839 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:43.147907 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:43.148034 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.148849 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.151518 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.152540 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.152600 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:43.152698 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:43.152975 (+ 277us) server_negotiation.cc:299] Negotiation successful
0924 23:14:43.153018 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":73,"threads_started":1}
I20260924 23:14:43.155534 31861 tablet_service.cc:1467] Processing CreateTablet for tablet c68d8ad86d514389b49455a709887c8e (DEFAULT_TABLE table=test-table [id=4e46a85b320d4e918465a51369fe40bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:43.155939 31861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c68d8ad86d514389b49455a709887c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:43.156656 32204 tablet_bootstrap.cc:492] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc: Bootstrap starting.
I20260924 23:14:43.157352 32204 tablet_bootstrap.cc:654] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:43.157696 32204 log.cc:824] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:43.158067 32203 tablet_bootstrap.cc:492] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d: Bootstrap starting.
I20260924 23:14:43.158514 32204 tablet_bootstrap.cc:492] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc: No bootstrap required, opened a new log
I20260924 23:14:43.158561 32204 ts_tablet_manager.cc:1397] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:14:43.158766 32203 tablet_bootstrap.cc:654] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:43.159071 32203 log.cc:824] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:43.159319 32206 tablet_bootstrap.cc:492] T c68d8ad86d514389b49455a709887c8e P bd59cdf3e31b4946a2bb39883ada8797: Bootstrap starting.
I20260924 23:14:43.159963 32206 tablet_bootstrap.cc:654] T c68d8ad86d514389b49455a709887c8e P bd59cdf3e31b4946a2bb39883ada8797: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:43.160293 32206 log.cc:824] T c68d8ad86d514389b49455a709887c8e P bd59cdf3e31b4946a2bb39883ada8797: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:43.161093 32206 tablet_bootstrap.cc:492] T c68d8ad86d514389b49455a709887c8e P bd59cdf3e31b4946a2bb39883ada8797: No bootstrap required, opened a new log
I20260924 23:14:43.161167 32206 ts_tablet_manager.cc:1397] T c68d8ad86d514389b49455a709887c8e P bd59cdf3e31b4946a2bb39883ada8797: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:14:43.161106 32204 raft_consensus.cc:348] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc036ee4e2c409481f108bae0290abc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44525 } } peers { permanent_uuid: "bd59cdf3e31b4946a2bb39883ada8797" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41273 } } peers { permanent_uuid: "0bdda79d65ad454f950585236c8e672d" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 43267 } }
I20260924 23:14:43.161285 32204 raft_consensus.cc:374] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:43.161325 32204 raft_consensus.cc:729] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccc036ee4e2c409481f108bae0290abc, State: Initialized, Role: FOLLOWER
I20260924 23:14:43.161443 32204 consensus_queue.cc:260] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc036ee4e2c409481f108bae0290abc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44525 } } peers { permanent_uuid: "bd59cdf3e31b4946a2bb39883ada8797" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41273 } } peers { permanent_uuid: "0bdda79d65ad454f950585236c8e672d" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 43267 } }
I20260924 23:14:43.161729 32204 ts_tablet_manager.cc:1428] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:14:43.161760 32058 heartbeater.cc:503] Master 127.12.212.62:44451 was elected leader, sending a full tablet report...
I20260924 23:14:43.162366 32203 tablet_bootstrap.cc:492] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d: No bootstrap required, opened a new log
I20260924 23:14:43.162429 32203 ts_tablet_manager.cc:1397] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260924 23:14:43.163666 32206 raft_consensus.cc:348] T c68d8ad86d514389b49455a709887c8e P bd59cdf3e31b4946a2bb39883ada8797 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc036ee4e2c409481f108bae0290abc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44525 } } peers { permanent_uuid: "bd59cdf3e31b4946a2bb39883ada8797" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41273 } } peers { permanent_uuid: "0bdda79d65ad454f950585236c8e672d" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 43267 } }
I20260924 23:14:43.163790 32206 raft_consensus.cc:374] T c68d8ad86d514389b49455a709887c8e P bd59cdf3e31b4946a2bb39883ada8797 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:43.163810 32206 raft_consensus.cc:729] T c68d8ad86d514389b49455a709887c8e P bd59cdf3e31b4946a2bb39883ada8797 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd59cdf3e31b4946a2bb39883ada8797, State: Initialized, Role: FOLLOWER
I20260924 23:14:43.163908 32206 consensus_queue.cc:260] T c68d8ad86d514389b49455a709887c8e P bd59cdf3e31b4946a2bb39883ada8797 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc036ee4e2c409481f108bae0290abc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44525 } } peers { permanent_uuid: "bd59cdf3e31b4946a2bb39883ada8797" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41273 } } peers { permanent_uuid: "0bdda79d65ad454f950585236c8e672d" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 43267 } }
I20260924 23:14:43.164165 32206 ts_tablet_manager.cc:1428] T c68d8ad86d514389b49455a709887c8e P bd59cdf3e31b4946a2bb39883ada8797: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:14:43.164172 31927 heartbeater.cc:503] Master 127.12.212.62:44451 was elected leader, sending a full tablet report...
I20260924 23:14:43.164655 32203 raft_consensus.cc:348] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc036ee4e2c409481f108bae0290abc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44525 } } peers { permanent_uuid: "bd59cdf3e31b4946a2bb39883ada8797" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41273 } } peers { permanent_uuid: "0bdda79d65ad454f950585236c8e672d" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 43267 } }
I20260924 23:14:43.164798 32203 raft_consensus.cc:374] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:43.164839 32203 raft_consensus.cc:729] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdda79d65ad454f950585236c8e672d, State: Initialized, Role: FOLLOWER
I20260924 23:14:43.164944 32203 consensus_queue.cc:260] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc036ee4e2c409481f108bae0290abc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44525 } } peers { permanent_uuid: "bd59cdf3e31b4946a2bb39883ada8797" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41273 } } peers { permanent_uuid: "0bdda79d65ad454f950585236c8e672d" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 43267 } }
I20260924 23:14:43.165194 32203 ts_tablet_manager.cc:1428] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260924 23:14:43.165235 32189 heartbeater.cc:503] Master 127.12.212.62:44451 was elected leader, sending a full tablet report...
W20260924 23:14:43.254900 32059 tablet.cc:2369] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:14:43.327092 32209 raft_consensus.cc:482] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:14:43.327243 32209 raft_consensus.cc:504] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc036ee4e2c409481f108bae0290abc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44525 } } peers { permanent_uuid: "bd59cdf3e31b4946a2bb39883ada8797" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41273 } } peers { permanent_uuid: "0bdda79d65ad454f950585236c8e672d" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 43267 } }
I20260924 23:14:43.327625 32209 leader_election.cc:288] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd59cdf3e31b4946a2bb39883ada8797 (127.12.212.1:41273), 0bdda79d65ad454f950585236c8e672d (127.12.212.3:43267)
I20260924 23:14:43.331013 32201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:43.327919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:43333 (local address 127.12.212.3:43267)
0924 23:14:43.327981 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:43.327983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:43.327994 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:14:43.328032 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:43.328033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:43.328040 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:43.328095 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:43.328252 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.328848 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.330116 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.330658 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.330766 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:43.330776 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:43.330923 (+ 147us) server_negotiation.cc:299] Negotiation successful
0924 23:14:43.330955 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 23:14:43.331032 32060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:43.327822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:43267 (local address 127.12.212.2:43333)
0924 23:14:43.327862 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:43.327874 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:43.327919 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:43.328114 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:43.328115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:43.328117 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:43.328188 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:43.328191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.328933 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.328934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:43.330048 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:43.330051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.330766 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.330768 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:43.330842 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:43.330846 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:14:43.330914 (+ 68us) client_negotiation.cc:240] Negotiation successful
0924 23:14:43.330959 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260924 23:14:43.331400 32143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c68d8ad86d514389b49455a709887c8e" candidate_uuid: "ccc036ee4e2c409481f108bae0290abc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdda79d65ad454f950585236c8e672d" is_pre_election: true
I20260924 23:14:43.331569 32143 raft_consensus.cc:2454] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccc036ee4e2c409481f108bae0290abc in term 0.
I20260924 23:14:43.331712 32202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:43.327928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:41023 (local address 127.12.212.1:41273)
0924 23:14:43.327981 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:43.327984 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:43.328212 (+ 228us) server_negotiation.cc:407] Connection header received
0924 23:14:43.328240 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:43.328241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:43.328254 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:43.328302 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:43.328489 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.328966 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.330870 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.331395 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.331450 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:43.331455 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:43.331609 (+ 154us) server_negotiation.cc:299] Negotiation successful
0924 23:14:43.331648 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 23:14:43.331724 32212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:43.327829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:41273 (local address 127.12.212.2:41023)
0924 23:14:43.328121 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:43.328129 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:43.328179 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:43.328331 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:43.328332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:43.328334 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:43.328404 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:43.328406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.329661 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.329663 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:43.330793 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:43.330796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.331472 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.331473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:43.331540 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:43.331544 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:14:43.331601 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 23:14:43.331648 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":136,"thread_start_us":92,"threads_started":1}
I20260924 23:14:43.331956 31947 leader_election.cc:302] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdda79d65ad454f950585236c8e672d, ccc036ee4e2c409481f108bae0290abc; no voters:
I20260924 23:14:43.332037 31881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c68d8ad86d514389b49455a709887c8e" candidate_uuid: "ccc036ee4e2c409481f108bae0290abc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd59cdf3e31b4946a2bb39883ada8797" is_pre_election: true
I20260924 23:14:43.332091 32209 raft_consensus.cc:2781] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:14:43.332132 32209 raft_consensus.cc:482] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:14:43.332149 32209 raft_consensus.cc:3037] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:43.332193 31881 raft_consensus.cc:2454] T c68d8ad86d514389b49455a709887c8e P bd59cdf3e31b4946a2bb39883ada8797 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccc036ee4e2c409481f108bae0290abc in term 0.
I20260924 23:14:43.332887 32209 raft_consensus.cc:504] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc036ee4e2c409481f108bae0290abc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44525 } } peers { permanent_uuid: "bd59cdf3e31b4946a2bb39883ada8797" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41273 } } peers { permanent_uuid: "0bdda79d65ad454f950585236c8e672d" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 43267 } }
I20260924 23:14:43.333005 32209 leader_election.cc:288] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [CANDIDATE]: Term 1 election: Requested vote from peers bd59cdf3e31b4946a2bb39883ada8797 (127.12.212.1:41273), 0bdda79d65ad454f950585236c8e672d (127.12.212.3:43267)
I20260924 23:14:43.333190 31881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c68d8ad86d514389b49455a709887c8e" candidate_uuid: "ccc036ee4e2c409481f108bae0290abc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd59cdf3e31b4946a2bb39883ada8797"
I20260924 23:14:43.333225 32143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c68d8ad86d514389b49455a709887c8e" candidate_uuid: "ccc036ee4e2c409481f108bae0290abc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdda79d65ad454f950585236c8e672d"
I20260924 23:14:43.333277 31881 raft_consensus.cc:3037] T c68d8ad86d514389b49455a709887c8e P bd59cdf3e31b4946a2bb39883ada8797 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:43.333290 32143 raft_consensus.cc:3037] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:43.334157 32143 raft_consensus.cc:2454] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccc036ee4e2c409481f108bae0290abc in term 1.
I20260924 23:14:43.334160 31881 raft_consensus.cc:2454] T c68d8ad86d514389b49455a709887c8e P bd59cdf3e31b4946a2bb39883ada8797 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccc036ee4e2c409481f108bae0290abc in term 1.
I20260924 23:14:43.334370 31946 leader_election.cc:302] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd59cdf3e31b4946a2bb39883ada8797, ccc036ee4e2c409481f108bae0290abc; no voters:
I20260924 23:14:43.334481 32209 raft_consensus.cc:2781] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:43.334650 32209 raft_consensus.cc:686] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [term 1 LEADER]: Becoming Leader. State: Replica: ccc036ee4e2c409481f108bae0290abc, State: Running, Role: LEADER
I20260924 23:14:43.334777 32209 consensus_queue.cc:237] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc036ee4e2c409481f108bae0290abc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44525 } } peers { permanent_uuid: "bd59cdf3e31b4946a2bb39883ada8797" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41273 } } peers { permanent_uuid: "0bdda79d65ad454f950585236c8e672d" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 43267 } }
I20260924 23:14:43.335537 31739 catalog_manager.cc:5495] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc reported cstate change: term changed from 0 to 1, leader changed from <none> to ccc036ee4e2c409481f108bae0290abc (127.12.212.2). New cstate: current_term: 1 leader_uuid: "ccc036ee4e2c409481f108bae0290abc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc036ee4e2c409481f108bae0290abc" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 44525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd59cdf3e31b4946a2bb39883ada8797" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdda79d65ad454f950585236c8e672d" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 43267 } health_report { overall_health: UNKNOWN } } }
W20260924 23:14:43.373965 31928 tablet.cc:2369] T c68d8ad86d514389b49455a709887c8e P bd59cdf3e31b4946a2bb39883ada8797: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 23:14:43.378455 32190 tablet.cc:2369] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:14:43.380041 32200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:43.377312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54398 (local address 127.12.212.2:44525)
0924 23:14:43.377366 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:43.377368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:43.377376 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:14:43.377418 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:43.377419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:43.377439 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:43.377498 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:14:43.377671 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.378192 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.379064 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:43.379296 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:43.379355 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:43.379611 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:14:43.379715 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:14:43.379823 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:43.379921 (+ 98us) server_negotiation.cc:299] Negotiation successful
0924 23:14:43.379930 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 23:14:43.382035 31881 raft_consensus.cc:1261] T c68d8ad86d514389b49455a709887c8e P bd59cdf3e31b4946a2bb39883ada8797 [term 1 FOLLOWER]: Refusing update from remote peer ccc036ee4e2c409481f108bae0290abc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:14:43.382030 32143 raft_consensus.cc:1261] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d [term 1 FOLLOWER]: Refusing update from remote peer ccc036ee4e2c409481f108bae0290abc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:14:43.382402 32209 consensus_queue.cc:1035] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd59cdf3e31b4946a2bb39883ada8797" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 41273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:14:43.382546 32213 consensus_queue.cc:1035] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdda79d65ad454f950585236c8e672d" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:14:43.384691 32216 mvcc.cc:205] Tried to move back new op lower bound from 7333034735129600000 to 7333034734939779072. Current Snapshot: MvccSnapshot[applied={T|T < 7333034735129600000}]
I20260924 23:14:43.385164 32217 mvcc.cc:205] Tried to move back new op lower bound from 7333034735129600000 to 7333034734939779072. Current Snapshot: MvccSnapshot[applied={T|T < 7333034735129600000}]
I20260924 23:14:43.387218 31738 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:14:43.387328 31738 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:14:43.388178 31738 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68838c6d8238472fb5b81e5865361ae7: Sending DeleteTablet for 3 replicas of tablet c68d8ad86d514389b49455a709887c8e
I20260924 23:14:43.388504 31861 tablet_service.cc:1514] Processing DeleteTablet for tablet c68d8ad86d514389b49455a709887c8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:43 UTC) from {username='slave'} at 127.0.0.1:46582
I20260924 23:14:43.388513 31992 tablet_service.cc:1514] Processing DeleteTablet for tablet c68d8ad86d514389b49455a709887c8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:43 UTC) from {username='slave'} at 127.0.0.1:54394
I20260924 23:14:43.388687 32123 tablet_service.cc:1514] Processing DeleteTablet for tablet c68d8ad86d514389b49455a709887c8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:43 UTC) from {username='slave'} at 127.0.0.1:57748
I20260924 23:14:43.388813 32227 tablet_replica.cc:330] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc: stopping tablet replica
I20260924 23:14:43.388911 32228 tablet_replica.cc:330] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d: stopping tablet replica
I20260924 23:14:43.389019 32227 raft_consensus.cc:2229] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:14:43.389040 32228 raft_consensus.cc:2229] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:14:43.389153 32228 raft_consensus.cc:2258] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:14:43.389235 32227 raft_consensus.cc:2258] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:14:43.389273 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 31799
I20260924 23:14:43.390226 32227 ts_tablet_manager.cc:1905] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:14:43.392040 32227 ts_tablet_manager.cc:1918] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:14:43.392103 32227 log.cc:1196] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/c68d8ad86d514389b49455a709887c8e
I20260924 23:14:43.392325 32227 ts_tablet_manager.cc:1939] T c68d8ad86d514389b49455a709887c8e P ccc036ee4e2c409481f108bae0290abc: Deleting consensus metadata
I20260924 23:14:43.392705 31726 catalog_manager.cc:4841] TS ccc036ee4e2c409481f108bae0290abc (127.12.212.2:44525): tablet c68d8ad86d514389b49455a709887c8e (table test-table [id=4e46a85b320d4e918465a51369fe40bb]) successfully deleted
W20260924 23:14:43.393235 31724 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.212.1:41273 (error 108)
I20260924 23:14:43.393738 32199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:43.393621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:41273 (local address 127.0.0.1:46588)
0924 23:14:43.393665 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:43.393706 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.1:41273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260924 23:14:43.393756 32228 ts_tablet_manager.cc:1905] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 23:14:43.393862 31724 catalog_manager.cc:4584] TS bd59cdf3e31b4946a2bb39883ada8797 (127.12.212.1:41273): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c68d8ad86d514389b49455a709887c8e: Network error: Client connection negotiation failed: client connection to 127.12.212.1:41273: connect: Connection refused (error 111)
I20260924 23:14:43.394438 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 31930
I20260924 23:14:43.395757 32228 ts_tablet_manager.cc:1918] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:14:43.395843 32228 log.cc:1196] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/c68d8ad86d514389b49455a709887c8e
I20260924 23:14:43.396067 32228 ts_tablet_manager.cc:1939] T c68d8ad86d514389b49455a709887c8e P 0bdda79d65ad454f950585236c8e672d: Deleting consensus metadata
I20260924 23:14:43.397365 31725 catalog_manager.cc:4841] TS 0bdda79d65ad454f950585236c8e672d (127.12.212.3:43267): tablet c68d8ad86d514389b49455a709887c8e (table test-table [id=4e46a85b320d4e918465a51369fe40bb]) successfully deleted
I20260924 23:14:43.400473 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 32061
I20260924 23:14:43.404786 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 31708
2026-09-24T23:14:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (841 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:14:43 dist-test-slave-m0z6 krb5kdc[32232](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:14:43 dist-test-slave-m0z6 krb5kdc[32232](info): set up 2 sockets
Sep 24 23:14:43 dist-test-slave-m0z6 krb5kdc[32232](info): commencing operation
krb5kdc: starting...
W20260924 23:14:44.358359 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:14:44 dist-test-slave-m0z6 krb5kdc[32232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:14:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:14:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:14:44.477128 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:42755
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:42753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:42755
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:44.556831 32245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:44.557055 32245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:44.557085 32245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:44.558256 32245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:14:44.558300 32245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:44.558317 32245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:44.558334 32245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:14:44.558349 32245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:14:44.559850 32245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:42753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:42755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:42755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:44.560096 32245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:44.560310 32245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:44.565129 32250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:44.565124 32253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:44.565168 32251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:44.565392 32245 server_base.cc:1034] running on GCE node
I20260924 23:14:44.565634 32245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:44.565933 32245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:44.567070 32245 hybrid_clock.cc:648] HybridClock initialized: now 1790291684567048 us; error 36 us; skew 500 ppm
Sep 24 23:14:44 dist-test-slave-m0z6 krb5kdc[32232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291684, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:44.569473 32245 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:14:44.569864 32245 webserver.cc:460] Webserver started at http://127.12.212.62:44185/ using document root <none> and password file <none>
I20260924 23:14:44.570036 32245 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:44.570075 32245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:44.570163 32245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:44.571211 32245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "0f543fc40fd54e3893678481e2516a34"
format_stamp: "Formatted at 2026-09-24 23:14:44 on dist-test-slave-m0z6"
server_key: "d4040fb10b48e165477c5b7a4c3375a2"
server_key_iv: "17cadff943dec4d116b0eb1f82a1b9be"
server_key_version: "encryptionkey@0"
I20260924 23:14:44.571533 32245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "0f543fc40fd54e3893678481e2516a34"
format_stamp: "Formatted at 2026-09-24 23:14:44 on dist-test-slave-m0z6"
server_key: "d4040fb10b48e165477c5b7a4c3375a2"
server_key_iv: "17cadff943dec4d116b0eb1f82a1b9be"
server_key_version: "encryptionkey@0"
I20260924 23:14:44.572759 32245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:14:44.573467 32260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:44.573679 32245 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:44.573750 32245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "0f543fc40fd54e3893678481e2516a34"
format_stamp: "Formatted at 2026-09-24 23:14:44 on dist-test-slave-m0z6"
server_key: "d4040fb10b48e165477c5b7a4c3375a2"
server_key_iv: "17cadff943dec4d116b0eb1f82a1b9be"
server_key_version: "encryptionkey@0"
I20260924 23:14:44.573802 32245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:44.602018 32245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:44.604108 32245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:44.604229 32245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:44.608028 32245 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:42755
I20260924 23:14:44.608132 32312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:42755 every 8 connection(s)
I20260924 23:14:44.608413 32245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:14:44.609345 32313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:44.611822 32313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34: Bootstrap starting.
I20260924 23:14:44.612097 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 32245
I20260924 23:14:44.612195 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:14:44.612350 13136 external_mini_cluster.cc:1428] Setting key fe2e259b2162cb4f6d56715066195f88
I20260924 23:14:44.612411 32313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:44.612709 32313 log.cc:824] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:44.613411 32313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34: No bootstrap required, opened a new log
I20260924 23:14:44.615336 32313 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f543fc40fd54e3893678481e2516a34" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42755 } }
I20260924 23:14:44.615486 32313 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:44.615509 32313 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f543fc40fd54e3893678481e2516a34, State: Initialized, Role: FOLLOWER
I20260924 23:14:44.615658 32313 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f543fc40fd54e3893678481e2516a34" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42755 } }
I20260924 23:14:44.615726 32313 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:14:44.615748 32313 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:14:44.615794 32313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:44.616403 32313 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f543fc40fd54e3893678481e2516a34" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42755 } }
I20260924 23:14:44.616508 32313 leader_election.cc:302] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f543fc40fd54e3893678481e2516a34; no voters:
I20260924 23:14:44.616688 32313 leader_election.cc:288] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:14:44.616760 32318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:44.616932 32313 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:14:44.617000 32318 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34 [term 1 LEADER]: Becoming Leader. State: Replica: 0f543fc40fd54e3893678481e2516a34, State: Running, Role: LEADER
I20260924 23:14:44.617127 32318 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f543fc40fd54e3893678481e2516a34" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42755 } }
I20260924 23:14:44.617497 32318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f543fc40fd54e3893678481e2516a34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f543fc40fd54e3893678481e2516a34" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42755 } } }
I20260924 23:14:44.617568 32318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:44.617687 32318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f543fc40fd54e3893678481e2516a34. Latest consensus state: current_term: 1 leader_uuid: "0f543fc40fd54e3893678481e2516a34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f543fc40fd54e3893678481e2516a34" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42755 } } }
I20260924 23:14:44.617731 32318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34 [sys.catalog]: This master's current role is: LEADER
W20260924 23:14:44.618813 32333 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:14:44.619261 32333 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 24 23:14:44 dist-test-slave-m0z6 krb5kdc[32232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:14:44.619397 32327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:14:44.619568 32327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:14:44.621088 32335 mvcc.cc:205] Tried to move back new op lower bound from 7333034740205445120 to 7333034740192186368. Current Snapshot: MvccSnapshot[applied={T|T < 7333034740205445120}]
I20260924 23:14:44.621515 32327 catalog_manager.cc:1347] Generated new cluster ID: 87bcc14b39374888975ef1741e0f9f57
I20260924 23:14:44.621565 32327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:14:44.623411 32317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:44.612851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34500 (local address 127.12.212.62:42755)
0924 23:14:44.613009 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:44.613019 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:44.613056 (+ 37us) server_negotiation.cc:407] Connection header received
0924 23:14:44.613557 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:44.613565 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:44.613618 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:44.613748 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:44.614395 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:44.615365 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:44.616327 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:44.616993 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:44.617504 (+ 511us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:44.619949 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:44.619954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:44.619954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:44.619967 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:44.621891 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:44.622236 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:44.622239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:44.622240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:44.622331 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:44.622508 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:44.622509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:44.622509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:44.622806 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:44.622961 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:44.623041 (+ 80us) server_negotiation.cc:299] Negotiation successful
0924 23:14:44.623193 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260924 23:14:44.636576 32327 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:14:44.637372 32327 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:14:44.642578 32327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34: Generated new TSK 0
I20260924 23:14:44.642853 32327 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.212.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
Entry for principal kudu/127.12.212.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab.
I20260924 23:14:44.675594 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42755
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:42753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:44.756359 32339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:44.756585 32339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:44.756609 32339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:44.757843 32339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:44.757890 32339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:44.757916 32339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:14:44.759418 32339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:42753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.1.keytab
--principal=kudu/127.12.212.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:44.759671 32339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:44.759907 32339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:14:44.765174 32339 server_base.cc:1034] running on GCE node
W20260924 23:14:44.765105 32344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:44.765136 32347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:44.765105 32345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:44.765582 32339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:44.765803 32339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:44.766947 32339 hybrid_clock.cc:648] HybridClock initialized: now 1790291684766932 us; error 29 us; skew 500 ppm
Sep 24 23:14:44 dist-test-slave-m0z6 krb5kdc[32232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291684, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:44.769688 32339 init.cc:377] Logged in from keytab as kudu/127.12.212.1@KRBTEST.COM (short username kudu)
I20260924 23:14:44.770171 32339 webserver.cc:460] Webserver started at http://127.12.212.1:45053/ using document root <none> and password file <none>
I20260924 23:14:44.770392 32339 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:44.770443 32339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:44.770567 32339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:44.771714 32339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "9fca4e410af34836a55b0f3f75362c7b"
format_stamp: "Formatted at 2026-09-24 23:14:44 on dist-test-slave-m0z6"
server_key: "7e3ed9c48e5190af2d5340a204ea002d"
server_key_iv: "d9fa40d4354fb8f92e83fd5a5e4a7bd1"
server_key_version: "encryptionkey@0"
I20260924 23:14:44.772053 32339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "9fca4e410af34836a55b0f3f75362c7b"
format_stamp: "Formatted at 2026-09-24 23:14:44 on dist-test-slave-m0z6"
server_key: "7e3ed9c48e5190af2d5340a204ea002d"
server_key_iv: "d9fa40d4354fb8f92e83fd5a5e4a7bd1"
server_key_version: "encryptionkey@0"
I20260924 23:14:44.773365 32339 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:14:44.774235 32354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:44.774410 32339 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:44.774503 32339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "9fca4e410af34836a55b0f3f75362c7b"
format_stamp: "Formatted at 2026-09-24 23:14:44 on dist-test-slave-m0z6"
server_key: "7e3ed9c48e5190af2d5340a204ea002d"
server_key_iv: "d9fa40d4354fb8f92e83fd5a5e4a7bd1"
server_key_version: "encryptionkey@0"
I20260924 23:14:44.774576 32339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:44.791988 32339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:44.793946 32339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:44.794080 32339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:44.794304 32339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:44.794600 32339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:44.794632 32339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:44.794659 32339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:44.794677 32339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:44.799988 32339 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:44979
I20260924 23:14:44.800057 32467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:44979 every 8 connection(s)
I20260924 23:14:44.800369 32339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
Sep 24 23:14:44 dist-test-slave-m0z6 krb5kdc[32232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291684, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.1@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:14:44.810086 32317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:44.801077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:42435 (local address 127.12.212.62:42755)
0924 23:14:44.801169 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:44.801172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:44.801331 (+ 159us) server_negotiation.cc:407] Connection header received
0924 23:14:44.801883 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:44.801884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:44.801897 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:44.801953 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:44.802940 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:44.803498 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:44.804524 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:44.804736 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:44.804791 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:44.807202 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:44.807204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:44.807204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:44.807212 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:44.808604 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:44.809103 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:44.809104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:44.809104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:44.809166 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:44.809415 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:44.809416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:44.809416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:44.809539 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:44.809597 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:44.809887 (+ 290us) server_negotiation.cc:299] Negotiation successful
0924 23:14:44.809990 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260924 23:14:44.810278 32470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:44.801039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42755 (local address 127.12.212.1:42435)
0924 23:14:44.801218 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:44.801259 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:44.801784 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:44.802031 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:44.802035 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:44.802596 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:44.802847 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:44.802853 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:44.803635 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:44.803635 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:44.804434 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:44.804436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:44.804865 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:44.804866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:44.804999 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:44.805563 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:14:44.805572 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:44.807070 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:44.808729 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:44.808742 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:44.808745 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:44.809021 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:44.809230 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:44.809231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:44.809231 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:44.809354 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:44.809594 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:44.809602 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:44.809762 (+ 160us) client_negotiation.cc:769] Sending connection context
0924 23:14:44.809859 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 23:14:44.810040 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260924 23:14:44.810540 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 32339
I20260924 23:14:44.810655 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:14:44.810883 13136 external_mini_cluster.cc:1428] Setting key 5414f3eea47bba8507796a882ec02a07
I20260924 23:14:44.810957 32468 heartbeater.cc:347] Connected to a master server at 127.12.212.62:42755
I20260924 23:14:44.811084 32468 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:44.811290 32468 heartbeater.cc:511] Master 127.12.212.62:42755 requested a full tablet report, sending...
I20260924 23:14:44.811810 32277 ts_manager.cc:194] Registered new tserver with Master: 9fca4e410af34836a55b0f3f75362c7b (127.12.212.1:44979)
I20260924 23:14:44.812422 32277 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.1@KRBTEST.COM'} at 127.12.212.1:42435
WARNING: no policy specified for kudu/127.12.212.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
Entry for principal kudu/127.12.212.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab.
I20260924 23:14:44.835276 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42755
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:42753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:44.916855 32473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:44.917084 32473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:44.917126 32473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:44.918452 32473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:44.918497 32473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:44.918535 32473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:14:44.920089 32473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:42753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.2.keytab
--principal=kudu/127.12.212.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:44.920341 32473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:44.920555 32473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:44.925923 32478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:44.925936 32481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:44.925916 32479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:44.926131 32473 server_base.cc:1034] running on GCE node
I20260924 23:14:44.926339 32473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:44.926576 32473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:44.927742 32473 hybrid_clock.cc:648] HybridClock initialized: now 1790291684927729 us; error 30 us; skew 500 ppm
Sep 24 23:14:44 dist-test-slave-m0z6 krb5kdc[32232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291684, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:44.930298 32473 init.cc:377] Logged in from keytab as kudu/127.12.212.2@KRBTEST.COM (short username kudu)
I20260924 23:14:44.930724 32473 webserver.cc:460] Webserver started at http://127.12.212.2:45711/ using document root <none> and password file <none>
I20260924 23:14:44.930912 32473 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:44.930953 32473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:44.931043 32473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:44.932050 32473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "dd17cf7cecc642eba653a74a5a3bd3ba"
format_stamp: "Formatted at 2026-09-24 23:14:44 on dist-test-slave-m0z6"
server_key: "8ec8d62dad00c6058874b757cdef5c8d"
server_key_iv: "ee852d582a87b826a22435eca9150167"
server_key_version: "encryptionkey@0"
I20260924 23:14:44.932337 32473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "dd17cf7cecc642eba653a74a5a3bd3ba"
format_stamp: "Formatted at 2026-09-24 23:14:44 on dist-test-slave-m0z6"
server_key: "8ec8d62dad00c6058874b757cdef5c8d"
server_key_iv: "ee852d582a87b826a22435eca9150167"
server_key_version: "encryptionkey@0"
I20260924 23:14:44.933491 32473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:14:44.934139 32488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:44.934300 32473 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:14:44.934417 32473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "dd17cf7cecc642eba653a74a5a3bd3ba"
format_stamp: "Formatted at 2026-09-24 23:14:44 on dist-test-slave-m0z6"
server_key: "8ec8d62dad00c6058874b757cdef5c8d"
server_key_iv: "ee852d582a87b826a22435eca9150167"
server_key_version: "encryptionkey@0"
I20260924 23:14:44.934505 32473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:44.984498 32473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:44.986847 32473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:44.986991 32473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:44.987202 32473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:44.987514 32473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:44.987547 32473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:44.987571 32473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:44.987592 32473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:44.992944 32473 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:37985
I20260924 23:14:44.993005 32601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:37985 every 8 connection(s)
I20260924 23:14:44.993322 32473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
Sep 24 23:14:44 dist-test-slave-m0z6 krb5kdc[32232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291684, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.2@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:14:45.000560 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 32473
I20260924 23:14:45.000667 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:14:45.000846 13136 external_mini_cluster.cc:1428] Setting key a4e2fc07872aec2fa25e9d7de7c576a7
I20260924 23:14:45.002779 32317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:44.993990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:49433 (local address 127.12.212.62:42755)
0924 23:14:44.994033 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:44.994035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:44.994252 (+ 217us) server_negotiation.cc:407] Connection header received
0924 23:14:44.994773 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:44.994774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:44.994788 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:44.994847 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:44.995819 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:44.996318 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:44.997289 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:44.997479 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:44.997534 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:44.999855 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:44.999857 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:44.999857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:44.999865 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:45.001268 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:45.001726 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:45.001727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:45.001727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:45.001782 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:45.002042 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:45.002043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:45.002043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:45.002153 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:45.002198 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:45.002568 (+ 370us) server_negotiation.cc:299] Negotiation successful
0924 23:14:45.002651 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 23:14:45.002962 32604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:44.993976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42755 (local address 127.12.212.2:49433)
0924 23:14:44.994122 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:44.994165 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:44.994656 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:44.994924 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:44.994928 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:44.995509 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:44.995716 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:44.995722 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:44.996403 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:44.996405 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:44.997207 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:44.997209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:44.997549 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:44.997550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:44.997680 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:44.998204 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:14:44.998213 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:44.999757 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:45.001339 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:45.001348 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:45.001351 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:45.001648 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:45.001843 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:45.001844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:45.001844 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:45.001984 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:45.002206 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:45.002213 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:45.002390 (+ 177us) client_negotiation.cc:769] Sending connection context
0924 23:14:45.002517 (+ 127us) client_negotiation.cc:240] Negotiation successful
0924 23:14:45.002719 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
I20260924 23:14:45.003446 32602 heartbeater.cc:347] Connected to a master server at 127.12.212.62:42755
I20260924 23:14:45.003533 32602 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:45.003702 32602 heartbeater.cc:511] Master 127.12.212.62:42755 requested a full tablet report, sending...
I20260924 23:14:45.004104 32277 ts_manager.cc:194] Registered new tserver with Master: dd17cf7cecc642eba653a74a5a3bd3ba (127.12.212.2:37985)
I20260924 23:14:45.004534 32277 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.2@KRBTEST.COM'} at 127.12.212.2:49433
WARNING: no policy specified for kudu/127.12.212.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
Entry for principal kudu/127.12.212.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab.
I20260924 23:14:45.025453 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42755
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:42753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:45.106352 32607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:45.106580 32607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:45.106612 32607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:45.107844 32607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:45.107892 32607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:45.107937 32607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:14:45.109436 32607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:42753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.3.keytab
--principal=kudu/127.12.212.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:42755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:45.109668 32607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:45.109879 32607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:45.114833 32613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:45.114835 32615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:45.114833 32612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:45.114861 32607 server_base.cc:1034] running on GCE node
I20260924 23:14:45.115216 32607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:45.115459 32607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:45.116623 32607 hybrid_clock.cc:648] HybridClock initialized: now 1790291685116602 us; error 35 us; skew 500 ppm
Sep 24 23:14:45 dist-test-slave-m0z6 krb5kdc[32232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291685, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:45.119429 32607 init.cc:377] Logged in from keytab as kudu/127.12.212.3@KRBTEST.COM (short username kudu)
I20260924 23:14:45.119889 32607 webserver.cc:460] Webserver started at http://127.12.212.3:44717/ using document root <none> and password file <none>
I20260924 23:14:45.120091 32607 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:45.120139 32607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:45.120265 32607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:45.121335 32607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "4663d9ed9b42441881a257aa6024bdf0"
format_stamp: "Formatted at 2026-09-24 23:14:45 on dist-test-slave-m0z6"
server_key: "1376e27aedfe4916a591a8ba11a3513d"
server_key_iv: "5eabf27357671704219aa9d863714447"
server_key_version: "encryptionkey@0"
I20260924 23:14:45.121661 32607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "4663d9ed9b42441881a257aa6024bdf0"
format_stamp: "Formatted at 2026-09-24 23:14:45 on dist-test-slave-m0z6"
server_key: "1376e27aedfe4916a591a8ba11a3513d"
server_key_iv: "5eabf27357671704219aa9d863714447"
server_key_version: "encryptionkey@0"
I20260924 23:14:45.123054 32607 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:14:45.123780 32622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:45.123971 32607 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:14:45.124068 32607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "4663d9ed9b42441881a257aa6024bdf0"
format_stamp: "Formatted at 2026-09-24 23:14:45 on dist-test-slave-m0z6"
server_key: "1376e27aedfe4916a591a8ba11a3513d"
server_key_iv: "5eabf27357671704219aa9d863714447"
server_key_version: "encryptionkey@0"
I20260924 23:14:45.124130 32607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:45.140275 32607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:45.142500 32607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:45.142812 32607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:45.143039 32607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:45.143519 32607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:45.143563 32607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:45.143589 32607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:45.143610 32607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:45.149240 32607 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:46027
I20260924 23:14:45.149307 32735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:46027 every 8 connection(s)
I20260924 23:14:45.149591 32607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:14:45.150177 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 32607
I20260924 23:14:45.150259 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:14:45.150436 13136 external_mini_cluster.cc:1428] Setting key 395cc850c7d4633c8fbb82903b897b17
Sep 24 23:14:45 dist-test-slave-m0z6 krb5kdc[32232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291685, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.3@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:14:45.158735 32317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.150322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:46357 (local address 127.12.212.62:42755)
0924 23:14:45.150390 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:45.150392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:45.150656 (+ 264us) server_negotiation.cc:407] Connection header received
0924 23:14:45.151154 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:45.151155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:45.151166 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:45.151227 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:45.152218 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.152723 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.153726 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.153922 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.153978 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.156158 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:45.156159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:45.156160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:45.156166 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:45.157507 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:45.157862 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:45.157862 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:45.157863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:45.157910 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:45.158089 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:45.158089 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:45.158089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:45.158230 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:45.158279 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:45.158566 (+ 287us) server_negotiation.cc:299] Negotiation successful
0924 23:14:45.158633 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 23:14:45.158951 32738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.150299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:42755 (local address 127.12.212.3:46357)
0924 23:14:45.150445 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:45.150487 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:45.151051 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:45.151307 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:45.151312 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:45.151911 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:45.152131 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.152136 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.152826 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.152827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.153642 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.153644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.153987 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.153988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.154089 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.154573 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:14:45.154581 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:45.156011 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:45.157580 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:45.157586 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:45.157589 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:45.157807 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:45.157955 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:14:45.157956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:14:45.157956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:14:45.158049 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:14:45.158304 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:45.158311 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:45.158441 (+ 130us) client_negotiation.cc:769] Sending connection context
0924 23:14:45.158547 (+ 106us) client_negotiation.cc:240] Negotiation successful
0924 23:14:45.158680 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
I20260924 23:14:45.159428 32736 heartbeater.cc:347] Connected to a master server at 127.12.212.62:42755
I20260924 23:14:45.159524 32736 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:45.159678 32736 heartbeater.cc:511] Master 127.12.212.62:42755 requested a full tablet report, sending...
I20260924 23:14:45.160000 32277 ts_manager.cc:194] Registered new tserver with Master: 4663d9ed9b42441881a257aa6024bdf0 (127.12.212.3:46027)
I20260924 23:14:45.160354 32277 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.212.3@KRBTEST.COM'} at 127.12.212.3:46357
I20260924 23:14:45.162081 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 23:14:45.169709 32317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.163444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34508 (local address 127.12.212.62:42755)
0924 23:14:45.163538 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:45.163541 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:45.163623 (+ 82us) server_negotiation.cc:407] Connection header received
0924 23:14:45.163690 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:45.163691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:45.163702 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:45.163751 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:45.164291 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.164793 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.165531 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.165724 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.165781 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.167092 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:45.167093 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:45.167094 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:45.167101 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:45.168534 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:45.168884 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:45.168886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:45.168886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:45.168940 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:45.169137 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:45.169138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:45.169138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:45.169300 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:45.169393 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:45.169515 (+ 122us) server_negotiation.cc:299] Negotiation successful
0924 23:14:45.169602 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 23:14:45.171311 32277 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:14:45.172895 32277 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:14:45.179324 32749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.175228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53260 (local address 127.12.212.2:37985)
0924 23:14:45.175637 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:45.175642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:45.175652 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:14:45.175691 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:45.175693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:45.175707 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:45.175782 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:45.175906 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.177026 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.178396 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.178955 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.179017 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.179078 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:45.179211 (+ 133us) server_negotiation.cc:299] Negotiation successful
0924 23:14:45.179250 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":75,"threads_started":1}
I20260924 23:14:45.179320 32747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.175099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:37985 (local address 127.0.0.1:53260)
0924 23:14:45.175309 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:45.175343 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:45.175401 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:45.175780 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:45.175781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:45.175783 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:45.175833 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.175835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.177154 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.177155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.178307 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.178309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.179034 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.179035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.179112 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.179125 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 23:14:45.179191 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 23:14:45.179244 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":42,"thread_start_us":138,"threads_started":1}
I20260924 23:14:45.179476 32748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.175088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:46027 (local address 127.0.0.1:42702)
0924 23:14:45.175293 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:45.175338 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:45.175398 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:45.175742 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:45.175745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:45.175748 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:45.175832 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.175834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.176964 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.176965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.178302 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.178305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.179251 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.179252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.179320 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.179324 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:14:45.179352 (+ 28us) client_negotiation.cc:240] Negotiation successful
0924 23:14:45.179392 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":2,"thread_start_us":65,"threads_started":1}
I20260924 23:14:45.179891 32750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.175298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:44979 (local address 127.0.0.1:49350)
0924 23:14:45.175408 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:45.175417 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:45.175459 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:45.176166 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:45.176166 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:45.176168 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:45.176201 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.176202 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.177418 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.177419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.178319 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.178322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.179304 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.179305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.179362 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.179365 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:14:45.179786 (+ 421us) client_negotiation.cc:240] Negotiation successful
0924 23:14:45.179831 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":77,"threads_started":1}
I20260924 23:14:45.179903 32752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.175588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49350 (local address 127.12.212.1:44979)
0924 23:14:45.175931 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:45.175935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:45.175944 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:14:45.175980 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:45.175983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:45.175996 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:45.176063 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:45.176279 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.177340 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.178405 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.179159 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.179221 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.179278 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:45.179808 (+ 530us) server_negotiation.cc:299] Negotiation successful
0924 23:14:45.179846 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":95,"threads_started":1}
I20260924 23:14:45.180114 32536 tablet_service.cc:1467] Processing CreateTablet for tablet 9a320ab6629546a6a4d9087cdcaaef6f (DEFAULT_TABLE table=test-table [id=371ffdbaaa6545e5b53301f644f5c12f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:45.180428 32536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a320ab6629546a6a4d9087cdcaaef6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:45.183100 32753 tablet_bootstrap.cc:492] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba: Bootstrap starting.
I20260924 23:14:45.183394 32751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.175444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42702 (local address 127.12.212.3:46027)
0924 23:14:45.175586 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:45.175591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:45.175600 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:14:45.175631 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:45.175633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:45.175645 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:45.175907 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:45.175980 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.176864 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.178384 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.179155 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.183149 (+ 3994us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.183230 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:45.183259 (+ 29us) server_negotiation.cc:299] Negotiation successful
0924 23:14:45.183312 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
I20260924 23:14:45.183580 32402 tablet_service.cc:1467] Processing CreateTablet for tablet 9a320ab6629546a6a4d9087cdcaaef6f (DEFAULT_TABLE table=test-table [id=371ffdbaaa6545e5b53301f644f5c12f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:45.183812 32753 tablet_bootstrap.cc:654] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:45.183956 32402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a320ab6629546a6a4d9087cdcaaef6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:45.184043 32670 tablet_service.cc:1467] Processing CreateTablet for tablet 9a320ab6629546a6a4d9087cdcaaef6f (DEFAULT_TABLE table=test-table [id=371ffdbaaa6545e5b53301f644f5c12f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:45.184340 32670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a320ab6629546a6a4d9087cdcaaef6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:45.184346 32753 log.cc:824] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:45.185118 32753 tablet_bootstrap.cc:492] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba: No bootstrap required, opened a new log
I20260924 23:14:45.185199 32753 ts_tablet_manager.cc:1397] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:14:45.186959 32755 tablet_bootstrap.cc:492] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0: Bootstrap starting.
I20260924 23:14:45.187237 32756 tablet_bootstrap.cc:492] T 9a320ab6629546a6a4d9087cdcaaef6f P 9fca4e410af34836a55b0f3f75362c7b: Bootstrap starting.
I20260924 23:14:45.187166 32753 raft_consensus.cc:348] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4663d9ed9b42441881a257aa6024bdf0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46027 } } peers { permanent_uuid: "dd17cf7cecc642eba653a74a5a3bd3ba" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 37985 } } peers { permanent_uuid: "9fca4e410af34836a55b0f3f75362c7b" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 44979 } }
I20260924 23:14:45.187354 32753 raft_consensus.cc:374] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:45.187399 32753 raft_consensus.cc:729] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd17cf7cecc642eba653a74a5a3bd3ba, State: Initialized, Role: FOLLOWER
I20260924 23:14:45.187503 32753 consensus_queue.cc:260] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4663d9ed9b42441881a257aa6024bdf0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46027 } } peers { permanent_uuid: "dd17cf7cecc642eba653a74a5a3bd3ba" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 37985 } } peers { permanent_uuid: "9fca4e410af34836a55b0f3f75362c7b" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 44979 } }
I20260924 23:14:45.187764 32755 tablet_bootstrap.cc:654] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:45.187762 32753 ts_tablet_manager.cc:1428] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:14:45.187775 32602 heartbeater.cc:503] Master 127.12.212.62:42755 was elected leader, sending a full tablet report...
I20260924 23:14:45.187798 32756 tablet_bootstrap.cc:654] T 9a320ab6629546a6a4d9087cdcaaef6f P 9fca4e410af34836a55b0f3f75362c7b: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:45.188344 32755 log.cc:824] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:45.188344 32756 log.cc:824] T 9a320ab6629546a6a4d9087cdcaaef6f P 9fca4e410af34836a55b0f3f75362c7b: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:45.189116 32756 tablet_bootstrap.cc:492] T 9a320ab6629546a6a4d9087cdcaaef6f P 9fca4e410af34836a55b0f3f75362c7b: No bootstrap required, opened a new log
I20260924 23:14:45.189144 32755 tablet_bootstrap.cc:492] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0: No bootstrap required, opened a new log
I20260924 23:14:45.189178 32756 ts_tablet_manager.cc:1397] T 9a320ab6629546a6a4d9087cdcaaef6f P 9fca4e410af34836a55b0f3f75362c7b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:14:45.189213 32755 ts_tablet_manager.cc:1397] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:14:45.190582 32755 raft_consensus.cc:348] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4663d9ed9b42441881a257aa6024bdf0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46027 } } peers { permanent_uuid: "dd17cf7cecc642eba653a74a5a3bd3ba" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 37985 } } peers { permanent_uuid: "9fca4e410af34836a55b0f3f75362c7b" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 44979 } }
I20260924 23:14:45.190563 32756 raft_consensus.cc:348] T 9a320ab6629546a6a4d9087cdcaaef6f P 9fca4e410af34836a55b0f3f75362c7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4663d9ed9b42441881a257aa6024bdf0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46027 } } peers { permanent_uuid: "dd17cf7cecc642eba653a74a5a3bd3ba" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 37985 } } peers { permanent_uuid: "9fca4e410af34836a55b0f3f75362c7b" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 44979 } }
I20260924 23:14:45.190750 32755 raft_consensus.cc:374] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:45.190761 32756 raft_consensus.cc:374] T 9a320ab6629546a6a4d9087cdcaaef6f P 9fca4e410af34836a55b0f3f75362c7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:45.190794 32755 raft_consensus.cc:729] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4663d9ed9b42441881a257aa6024bdf0, State: Initialized, Role: FOLLOWER
I20260924 23:14:45.190801 32756 raft_consensus.cc:729] T 9a320ab6629546a6a4d9087cdcaaef6f P 9fca4e410af34836a55b0f3f75362c7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fca4e410af34836a55b0f3f75362c7b, State: Initialized, Role: FOLLOWER
I20260924 23:14:45.190907 32755 consensus_queue.cc:260] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4663d9ed9b42441881a257aa6024bdf0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46027 } } peers { permanent_uuid: "dd17cf7cecc642eba653a74a5a3bd3ba" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 37985 } } peers { permanent_uuid: "9fca4e410af34836a55b0f3f75362c7b" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 44979 } }
I20260924 23:14:45.190908 32756 consensus_queue.cc:260] T 9a320ab6629546a6a4d9087cdcaaef6f P 9fca4e410af34836a55b0f3f75362c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4663d9ed9b42441881a257aa6024bdf0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46027 } } peers { permanent_uuid: "dd17cf7cecc642eba653a74a5a3bd3ba" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 37985 } } peers { permanent_uuid: "9fca4e410af34836a55b0f3f75362c7b" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 44979 } }
I20260924 23:14:45.191183 32756 ts_tablet_manager.cc:1428] T 9a320ab6629546a6a4d9087cdcaaef6f P 9fca4e410af34836a55b0f3f75362c7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:14:45.191195 32736 heartbeater.cc:503] Master 127.12.212.62:42755 was elected leader, sending a full tablet report...
I20260924 23:14:45.191195 32468 heartbeater.cc:503] Master 127.12.212.62:42755 was elected leader, sending a full tablet report...
I20260924 23:14:45.191183 32755 ts_tablet_manager.cc:1428] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 23:14:45.236346 32757 raft_consensus.cc:482] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:14:45.236466 32757 raft_consensus.cc:504] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4663d9ed9b42441881a257aa6024bdf0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46027 } } peers { permanent_uuid: "dd17cf7cecc642eba653a74a5a3bd3ba" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 37985 } } peers { permanent_uuid: "9fca4e410af34836a55b0f3f75362c7b" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 44979 } }
I20260924 23:14:45.236768 32757 leader_election.cc:288] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4663d9ed9b42441881a257aa6024bdf0 (127.12.212.3:46027), 9fca4e410af34836a55b0f3f75362c7b (127.12.212.1:44979)
I20260924 23:14:45.239724 32751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.236991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:57039 (local address 127.12.212.3:46027)
0924 23:14:45.237045 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:45.237047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:45.237056 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:14:45.237147 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:45.237148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:45.237158 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:45.237203 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:45.237446 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.237958 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.239041 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.239408 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.239472 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.239481 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:45.239620 (+ 139us) server_negotiation.cc:299] Negotiation successful
0924 23:14:45.239657 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 23:14:45.239722 32752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.237181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:47791 (local address 127.12.212.1:44979)
0924 23:14:45.237232 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:45.237234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:45.237269 (+ 35us) server_negotiation.cc:407] Connection header received
0924 23:14:45.237318 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:45.237319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:45.237328 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:45.237378 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:45.237527 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.237995 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.239035 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.239403 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.239465 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.239474 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:45.239610 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 23:14:45.239652 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 23:14:45.239764 32604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.236931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:46027 (local address 127.12.212.2:57039)
0924 23:14:45.236987 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:45.236998 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:45.237050 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:45.237262 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:45.237263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:45.237266 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:45.237359 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.237362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.238041 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.238042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.238952 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.238954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.239484 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.239485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.239555 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.239558 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:14:45.239605 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 23:14:45.239645 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":65}
I20260924 23:14:45.239774 32762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.237028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:44979 (local address 127.12.212.2:47791)
0924 23:14:45.237209 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:45.237219 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:45.237276 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:45.237387 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:45.237388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:45.237389 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:45.237465 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.237467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.238070 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.238071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.238949 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.238952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.239480 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.239481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.239545 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.239549 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:14:45.239594 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 23:14:45.239646 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":37,"thread_start_us":108,"threads_started":1}
I20260924 23:14:45.240103 32422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a320ab6629546a6a4d9087cdcaaef6f" candidate_uuid: "dd17cf7cecc642eba653a74a5a3bd3ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fca4e410af34836a55b0f3f75362c7b" is_pre_election: true
I20260924 23:14:45.240095 32690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a320ab6629546a6a4d9087cdcaaef6f" candidate_uuid: "dd17cf7cecc642eba653a74a5a3bd3ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4663d9ed9b42441881a257aa6024bdf0" is_pre_election: true
I20260924 23:14:45.240252 32422 raft_consensus.cc:2454] T 9a320ab6629546a6a4d9087cdcaaef6f P 9fca4e410af34836a55b0f3f75362c7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd17cf7cecc642eba653a74a5a3bd3ba in term 0.
I20260924 23:14:45.240252 32690 raft_consensus.cc:2454] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd17cf7cecc642eba653a74a5a3bd3ba in term 0.
I20260924 23:14:45.240451 32492 leader_election.cc:302] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4663d9ed9b42441881a257aa6024bdf0, dd17cf7cecc642eba653a74a5a3bd3ba; no voters:
I20260924 23:14:45.240590 32757 raft_consensus.cc:2781] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:14:45.240666 32757 raft_consensus.cc:482] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:14:45.240682 32757 raft_consensus.cc:3037] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:45.241364 32757 raft_consensus.cc:504] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4663d9ed9b42441881a257aa6024bdf0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46027 } } peers { permanent_uuid: "dd17cf7cecc642eba653a74a5a3bd3ba" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 37985 } } peers { permanent_uuid: "9fca4e410af34836a55b0f3f75362c7b" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 44979 } }
I20260924 23:14:45.241475 32757 leader_election.cc:288] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [CANDIDATE]: Term 1 election: Requested vote from peers 4663d9ed9b42441881a257aa6024bdf0 (127.12.212.3:46027), 9fca4e410af34836a55b0f3f75362c7b (127.12.212.1:44979)
I20260924 23:14:45.241688 32690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a320ab6629546a6a4d9087cdcaaef6f" candidate_uuid: "dd17cf7cecc642eba653a74a5a3bd3ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4663d9ed9b42441881a257aa6024bdf0"
I20260924 23:14:45.241708 32422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a320ab6629546a6a4d9087cdcaaef6f" candidate_uuid: "dd17cf7cecc642eba653a74a5a3bd3ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fca4e410af34836a55b0f3f75362c7b"
I20260924 23:14:45.241756 32690 raft_consensus.cc:3037] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:45.241783 32422 raft_consensus.cc:3037] T 9a320ab6629546a6a4d9087cdcaaef6f P 9fca4e410af34836a55b0f3f75362c7b [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:45.242422 32690 raft_consensus.cc:2454] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd17cf7cecc642eba653a74a5a3bd3ba in term 1.
I20260924 23:14:45.242583 32492 leader_election.cc:302] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4663d9ed9b42441881a257aa6024bdf0, dd17cf7cecc642eba653a74a5a3bd3ba; no voters:
I20260924 23:14:45.242609 32422 raft_consensus.cc:2454] T 9a320ab6629546a6a4d9087cdcaaef6f P 9fca4e410af34836a55b0f3f75362c7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd17cf7cecc642eba653a74a5a3bd3ba in term 1.
I20260924 23:14:45.242684 32757 raft_consensus.cc:2781] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:45.242874 32757 raft_consensus.cc:686] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [term 1 LEADER]: Becoming Leader. State: Replica: dd17cf7cecc642eba653a74a5a3bd3ba, State: Running, Role: LEADER
I20260924 23:14:45.242969 32757 consensus_queue.cc:237] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4663d9ed9b42441881a257aa6024bdf0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46027 } } peers { permanent_uuid: "dd17cf7cecc642eba653a74a5a3bd3ba" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 37985 } } peers { permanent_uuid: "9fca4e410af34836a55b0f3f75362c7b" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 44979 } }
W20260924 23:14:45.243803 32603 tablet.cc:2369] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:14:45.243870 32277 catalog_manager.cc:5495] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba reported cstate change: term changed from 0 to 1, leader changed from <none> to dd17cf7cecc642eba653a74a5a3bd3ba (127.12.212.2). New cstate: current_term: 1 leader_uuid: "dd17cf7cecc642eba653a74a5a3bd3ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4663d9ed9b42441881a257aa6024bdf0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd17cf7cecc642eba653a74a5a3bd3ba" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 37985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9fca4e410af34836a55b0f3f75362c7b" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 44979 } health_report { overall_health: UNKNOWN } } }
I20260924 23:14:45.254940 32749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.252564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53272 (local address 127.12.212.2:37985)
0924 23:14:45.252606 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:45.252608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:45.252616 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:14:45.252685 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:45.252685 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:45.252705 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:45.252766 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:14:45.252861 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.253353 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.254070 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.254280 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.254330 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.254521 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:14:45.254630 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:14:45.254729 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:45.254837 (+ 108us) server_negotiation.cc:299] Negotiation successful
0924 23:14:45.254856 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 23:14:45.256680 32690 raft_consensus.cc:1261] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0 [term 1 FOLLOWER]: Refusing update from remote peer dd17cf7cecc642eba653a74a5a3bd3ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:14:45.256681 32422 raft_consensus.cc:1261] T 9a320ab6629546a6a4d9087cdcaaef6f P 9fca4e410af34836a55b0f3f75362c7b [term 1 FOLLOWER]: Refusing update from remote peer dd17cf7cecc642eba653a74a5a3bd3ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:14:45.257076 32763 consensus_queue.cc:1035] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "4663d9ed9b42441881a257aa6024bdf0" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 46027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:14:45.257228 32757 consensus_queue.cc:1035] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fca4e410af34836a55b0f3f75362c7b" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 44979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:14:45.259734 32767 mvcc.cc:205] Tried to move back new op lower bound from 7333034742808735744 to 7333034742755864576. Current Snapshot: MvccSnapshot[applied={T|T < 7333034742808735744}]
I20260924 23:14:45.259995 32766 mvcc.cc:205] Tried to move back new op lower bound from 7333034742808735744 to 7333034742755864576. Current Snapshot: MvccSnapshot[applied={T|T < 7333034742808735744}]
I20260924 23:14:45.261883 32277 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:14:45.261965 32277 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:14:45.262668 32277 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f543fc40fd54e3893678481e2516a34: Sending DeleteTablet for 3 replicas of tablet 9a320ab6629546a6a4d9087cdcaaef6f
I20260924 23:14:45.262997 32670 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a320ab6629546a6a4d9087cdcaaef6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:45 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:42702
I20260924 23:14:45.263053 32536 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a320ab6629546a6a4d9087cdcaaef6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:45 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:53260
I20260924 23:14:45.263054 32402 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a320ab6629546a6a4d9087cdcaaef6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:45 UTC) from {username='kudu', principal='kudu/127.12.212.62@KRBTEST.COM'} at 127.0.0.1:49350
I20260924 23:14:45.263296 310 tablet_replica.cc:330] T 9a320ab6629546a6a4d9087cdcaaef6f P 9fca4e410af34836a55b0f3f75362c7b: stopping tablet replica
I20260924 23:14:45.263405 310 raft_consensus.cc:2229] T 9a320ab6629546a6a4d9087cdcaaef6f P 9fca4e410af34836a55b0f3f75362c7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:14:45.263479 308 tablet_replica.cc:330] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0: stopping tablet replica
I20260924 23:14:45.263520 310 raft_consensus.cc:2258] T 9a320ab6629546a6a4d9087cdcaaef6f P 9fca4e410af34836a55b0f3f75362c7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:14:45.263561 308 raft_consensus.cc:2229] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:14:45.263581 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 32339
I20260924 23:14:45.263643 308 raft_consensus.cc:2258] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:14:45.263760 309 tablet_replica.cc:330] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba: stopping tablet replica
I20260924 23:14:45.263827 309 raft_consensus.cc:2229] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:14:45.263914 309 raft_consensus.cc:2258] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:14:45.264446 309 ts_tablet_manager.cc:1905] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:14:45.265868 309 ts_tablet_manager.cc:1918] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:14:45.265916 309 log.cc:1196] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/9a320ab6629546a6a4d9087cdcaaef6f
I20260924 23:14:45.266088 309 ts_tablet_manager.cc:1939] T 9a320ab6629546a6a4d9087cdcaaef6f P dd17cf7cecc642eba653a74a5a3bd3ba: Deleting consensus metadata
I20260924 23:14:45.266512 32261 catalog_manager.cc:4841] TS dd17cf7cecc642eba653a74a5a3bd3ba (127.12.212.2:37985): tablet 9a320ab6629546a6a4d9087cdcaaef6f (table test-table [id=371ffdbaaa6545e5b53301f644f5c12f]) successfully deleted
W20260924 23:14:45.269393 32264 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.212.1:44979 (error 108)
I20260924 23:14:45.269758 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 32473
I20260924 23:14:45.270337 308 ts_tablet_manager.cc:1905] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:14:45.271891 32750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.271730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:44979 (local address 127.0.0.1:49352)
0924 23:14:45.271794 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:45.271848 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.1:44979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260924 23:14:45.272035 32264 catalog_manager.cc:4584] TS 9fca4e410af34836a55b0f3f75362c7b (127.12.212.1:44979): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9a320ab6629546a6a4d9087cdcaaef6f: Network error: Client connection negotiation failed: client connection to 127.12.212.1:44979: connect: Connection refused (error 111)
I20260924 23:14:45.272038 308 ts_tablet_manager.cc:1918] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:14:45.272117 308 log.cc:1196] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/9a320ab6629546a6a4d9087cdcaaef6f
I20260924 23:14:45.272369 308 ts_tablet_manager.cc:1939] T 9a320ab6629546a6a4d9087cdcaaef6f P 4663d9ed9b42441881a257aa6024bdf0: Deleting consensus metadata
I20260924 23:14:45.272825 32264 catalog_manager.cc:4841] TS 4663d9ed9b42441881a257aa6024bdf0 (127.12.212.3:46027): tablet 9a320ab6629546a6a4d9087cdcaaef6f (table test-table [id=371ffdbaaa6545e5b53301f644f5c12f]) successfully deleted
I20260924 23:14:45.275009 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 32607
I20260924 23:14:45.279359 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 32245
2026-09-24T23:14:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1875 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-24T23:14:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:14:45Z Disabled control of system clock
I20260924 23:14:45.305954 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:39475
--webserver_interface=127.12.212.62
--webserver_port=0
--builtin_ntp_servers=127.12.212.20:41421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:39475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 23:14:45.387187 315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:45.387424 315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:45.387444 315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:45.388559 315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:14:45.388598 315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:45.388610 315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:45.388621 315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:14:45.388633 315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:14:45.389967 315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:41421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:39475
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:39475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:45.390197 315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:45.390455 315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:14:45.395118 315 server_base.cc:1034] running on GCE node
W20260924 23:14:45.395020 323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:45.395025 320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:45.395053 321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:45.395471 315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:45.395718 315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:45.396852 315 hybrid_clock.cc:648] HybridClock initialized: now 1790291685396841 us; error 26 us; skew 500 ppm
I20260924 23:14:45.398114 315 webserver.cc:460] Webserver started at http://127.12.212.62:43363/ using document root <none> and password file <none>
I20260924 23:14:45.398350 315 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:45.398396 315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:45.398530 315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:45.399917 315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "fb9ec23cfc34441bad52bc824cda814d"
format_stamp: "Formatted at 2026-09-24 23:14:45 on dist-test-slave-m0z6"
server_key: "e8044a550d71936cc4515231b0b365c9"
server_key_iv: "74deafeff9b5097f2e285ee0877536c2"
server_key_version: "encryptionkey@0"
I20260924 23:14:45.400353 315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "fb9ec23cfc34441bad52bc824cda814d"
format_stamp: "Formatted at 2026-09-24 23:14:45 on dist-test-slave-m0z6"
server_key: "e8044a550d71936cc4515231b0b365c9"
server_key_iv: "74deafeff9b5097f2e285ee0877536c2"
server_key_version: "encryptionkey@0"
I20260924 23:14:45.402205 315 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260924 23:14:45.403283 329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:45.403504 315 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 23:14:45.403602 315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "fb9ec23cfc34441bad52bc824cda814d"
format_stamp: "Formatted at 2026-09-24 23:14:45 on dist-test-slave-m0z6"
server_key: "e8044a550d71936cc4515231b0b365c9"
server_key_iv: "74deafeff9b5097f2e285ee0877536c2"
server_key_version: "encryptionkey@0"
I20260924 23:14:45.403726 315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:45.426855 315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:45.427181 315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:45.427318 315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:45.431192 315 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:39475
I20260924 23:14:45.431255 381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:39475 every 8 connection(s)
I20260924 23:14:45.431578 315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:14:45.432561 382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:45.435501 382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d: Bootstrap starting.
I20260924 23:14:45.436193 382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:45.436549 382 log.cc:824] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:45.437345 382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d: No bootstrap required, opened a new log
I20260924 23:14:45.439302 382 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9ec23cfc34441bad52bc824cda814d" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39475 } }
I20260924 23:14:45.439469 382 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:45.439491 382 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb9ec23cfc34441bad52bc824cda814d, State: Initialized, Role: FOLLOWER
I20260924 23:14:45.439640 382 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9ec23cfc34441bad52bc824cda814d" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39475 } }
I20260924 23:14:45.439733 382 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:14:45.439756 382 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:14:45.439790 382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:45.440338 382 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9ec23cfc34441bad52bc824cda814d" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39475 } }
I20260924 23:14:45.440431 382 leader_election.cc:302] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb9ec23cfc34441bad52bc824cda814d; no voters:
I20260924 23:14:45.440600 382 leader_election.cc:288] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:14:45.440681 385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:45.440886 385 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d [term 1 LEADER]: Becoming Leader. State: Replica: fb9ec23cfc34441bad52bc824cda814d, State: Running, Role: LEADER
I20260924 23:14:45.440912 382 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:14:45.440938 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 315
I20260924 23:14:45.440996 385 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9ec23cfc34441bad52bc824cda814d" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39475 } }
I20260924 23:14:45.441041 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:14:45.441254 13136 external_mini_cluster.cc:1428] Setting key c22e607f275bb946ee7b781b9a994fe3
I20260924 23:14:45.441576 387 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb9ec23cfc34441bad52bc824cda814d. Latest consensus state: current_term: 1 leader_uuid: "fb9ec23cfc34441bad52bc824cda814d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9ec23cfc34441bad52bc824cda814d" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39475 } } }
I20260924 23:14:45.441660 387 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:45.441982 386 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb9ec23cfc34441bad52bc824cda814d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9ec23cfc34441bad52bc824cda814d" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39475 } } }
I20260924 23:14:45.442055 386 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:45.442193 395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:14:45.442685 395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:14:45.444258 395 catalog_manager.cc:1347] Generated new cluster ID: cff101b819f2445b9b4c8a1597bbfe73
I20260924 23:14:45.444309 395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:14:45.449013 396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.441721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39590 (local address 127.12.212.62:39475)
0924 23:14:45.442335 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:45.442345 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:45.443094 (+ 749us) server_negotiation.cc:407] Connection header received
0924 23:14:45.443349 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:45.443356 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:45.443432 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:45.443821 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:45.443917 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.445338 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.446500 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.447212 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.447718 (+ 506us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.448089 (+ 371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:45.448093 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:45.448094 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:45.448324 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:45.448370 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:45.448386 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:45.448640 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:45.448677 (+ 37us) server_negotiation.cc:299] Negotiation successful
0924 23:14:45.448766 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":71,"threads_started":1}
I20260924 23:14:45.481822 395 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:14:45.482645 395 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:14:45.493801 395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d: Generated new TSK 0
I20260924 23:14:45.494048 395 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:14:45.501598 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.1:0
--local_ip_for_outbound_sockets=127.12.212.1
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:39475
--builtin_ntp_servers=127.12.212.20:41421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 23:14:45.583618 406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:45.583835 406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:45.583858 406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:45.585063 406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:45.585108 406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:45.585141 406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.1
I20260924 23:14:45.586534 406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:41421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.212.1
--webserver_port=0
--tserver_master_addrs=127.12.212.62:39475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.1
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:45.586786 406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:45.587064 406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:45.591962 412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:45.591964 411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:45.591996 406 server_base.cc:1034] running on GCE node
W20260924 23:14:45.591975 414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:45.592410 406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:45.592644 406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:45.593793 406 hybrid_clock.cc:648] HybridClock initialized: now 1790291685593778 us; error 28 us; skew 500 ppm
I20260924 23:14:45.595173 406 webserver.cc:460] Webserver started at http://127.12.212.1:36817/ using document root <none> and password file <none>
I20260924 23:14:45.595407 406 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:45.595451 406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:45.595574 406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:45.596637 406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-0/data/instance:
uuid: "62b8ff0d48c242c19f12d6d6a383f9c1"
format_stamp: "Formatted at 2026-09-24 23:14:45 on dist-test-slave-m0z6"
server_key: "557495786140a06f2640de580c6ef62c"
server_key_iv: "7912308501b6e979af45fb2978197629"
server_key_version: "encryptionkey@0"
I20260924 23:14:45.596967 406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance:
uuid: "62b8ff0d48c242c19f12d6d6a383f9c1"
format_stamp: "Formatted at 2026-09-24 23:14:45 on dist-test-slave-m0z6"
server_key: "557495786140a06f2640de580c6ef62c"
server_key_iv: "7912308501b6e979af45fb2978197629"
server_key_version: "encryptionkey@0"
I20260924 23:14:45.598400 406 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:14:45.599239 420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:45.599439 406 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 23:14:45.599529 406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-0/wal
uuid: "62b8ff0d48c242c19f12d6d6a383f9c1"
format_stamp: "Formatted at 2026-09-24 23:14:45 on dist-test-slave-m0z6"
server_key: "557495786140a06f2640de580c6ef62c"
server_key_iv: "7912308501b6e979af45fb2978197629"
server_key_version: "encryptionkey@0"
I20260924 23:14:45.599606 406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:45.630280 406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:45.630613 406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:45.630785 406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:45.631016 406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:45.631353 406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:45.631387 406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:45.631419 406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:45.631439 406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:45.636783 406 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.1:46311
I20260924 23:14:45.636852 533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.1:46311 every 8 connection(s)
I20260924 23:14:45.637183 406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-0/data/info.pb
I20260924 23:14:45.642678 396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.637965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.1:52559 (local address 127.12.212.62:39475)
0924 23:14:45.638025 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:45.638027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:45.638261 (+ 234us) server_negotiation.cc:407] Connection header received
0924 23:14:45.638740 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:45.638742 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:45.638765 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:45.638827 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:45.639518 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.640008 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.641171 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.641438 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.641499 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.642419 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:45.642420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:45.642421 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:45.642426 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:45.642446 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:45.642451 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:45.642513 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:45.642579 (+ 66us) server_negotiation.cc:299] Negotiation successful
0924 23:14:45.642609 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 23:14:45.642948 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.637906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:39475 (local address 127.12.212.1:52559)
0924 23:14:45.638106 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:45.638148 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:45.638632 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:45.638901 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:45.638905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:45.638914 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:45.639431 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.639436 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.640086 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.640087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.641073 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.641076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.641516 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.641517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.641654 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.642263 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:14:45.642264 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:45.642295 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:14:45.642320 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:45.642503 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:45.642508 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:45.642511 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:14:45.642579 (+ 68us) client_negotiation.cc:240] Negotiation successful
0924 23:14:45.642722 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260924 23:14:45.643455 534 heartbeater.cc:347] Connected to a master server at 127.12.212.62:39475
I20260924 23:14:45.643549 534 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:45.643709 534 heartbeater.cc:511] Master 127.12.212.62:39475 requested a full tablet report, sending...
I20260924 23:14:45.644199 346 ts_manager.cc:194] Registered new tserver with Master: 62b8ff0d48c242c19f12d6d6a383f9c1 (127.12.212.1:46311)
I20260924 23:14:45.644855 346 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.212.1:52559
I20260924 23:14:45.646596 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 406
I20260924 23:14:45.646723 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-0/wal/instance
I20260924 23:14:45.646915 13136 external_mini_cluster.cc:1428] Setting key 7f5ebf524b6a8a450c6af4722644dc06
I20260924 23:14:45.647851 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.2:0
--local_ip_for_outbound_sockets=127.12.212.2
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:39475
--builtin_ntp_servers=127.12.212.20:41421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 23:14:45.728652 537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:45.728860 537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:45.728881 537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:45.730032 537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:45.730069 537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:45.730093 537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.2
I20260924 23:14:45.731431 537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:41421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.212.2
--webserver_port=0
--tserver_master_addrs=127.12.212.62:39475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.2
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:45.731637 537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:45.731892 537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:45.737353 543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:45.737279 542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:45.737404 537 server_base.cc:1034] running on GCE node
W20260924 23:14:45.737268 545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:45.737821 537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:45.738103 537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:45.739300 537 hybrid_clock.cc:648] HybridClock initialized: now 1790291685739283 us; error 36 us; skew 500 ppm
I20260924 23:14:45.740881 537 webserver.cc:460] Webserver started at http://127.12.212.2:42555/ using document root <none> and password file <none>
I20260924 23:14:45.741195 537 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:45.741272 537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:45.741420 537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:45.742929 537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-1/data/instance:
uuid: "0f442631ba9443638f45397461017cd1"
format_stamp: "Formatted at 2026-09-24 23:14:45 on dist-test-slave-m0z6"
server_key: "29429ab36a340479d8d8d726b00f47bf"
server_key_iv: "c83de76b76911b50675c30c77a52f8ae"
server_key_version: "encryptionkey@0"
I20260924 23:14:45.743363 537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance:
uuid: "0f442631ba9443638f45397461017cd1"
format_stamp: "Formatted at 2026-09-24 23:14:45 on dist-test-slave-m0z6"
server_key: "29429ab36a340479d8d8d726b00f47bf"
server_key_iv: "c83de76b76911b50675c30c77a52f8ae"
server_key_version: "encryptionkey@0"
I20260924 23:14:45.744751 537 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:14:45.745679 551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:45.745891 537 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:45.745988 537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-1/wal
uuid: "0f442631ba9443638f45397461017cd1"
format_stamp: "Formatted at 2026-09-24 23:14:45 on dist-test-slave-m0z6"
server_key: "29429ab36a340479d8d8d726b00f47bf"
server_key_iv: "c83de76b76911b50675c30c77a52f8ae"
server_key_version: "encryptionkey@0"
I20260924 23:14:45.746055 537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:45.757768 537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:45.758121 537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:45.758262 537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:45.758512 537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:45.758879 537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:45.758914 537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:45.758948 537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:45.758968 537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:45.764254 537 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.2:46533
I20260924 23:14:45.764323 664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.2:46533 every 8 connection(s)
I20260924 23:14:45.764629 537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-1/data/info.pb
I20260924 23:14:45.770272 396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.765381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:34627 (local address 127.12.212.62:39475)
0924 23:14:45.765438 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:45.765440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:45.765673 (+ 233us) server_negotiation.cc:407] Connection header received
0924 23:14:45.766161 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:45.766162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:45.766192 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:45.766257 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:45.766994 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.767509 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.768789 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.769003 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.769123 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.770016 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:45.770018 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:45.770018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:45.770024 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:45.770040 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:45.770044 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:45.770097 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:45.770171 (+ 74us) server_negotiation.cc:299] Negotiation successful
0924 23:14:45.770202 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 23:14:45.770535 667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.765327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:39475 (local address 127.12.212.2:34627)
0924 23:14:45.765509 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:45.765552 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:45.766042 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:45.766343 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:45.766347 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:45.766356 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:45.766890 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.766895 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.767623 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.767624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.768668 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.768672 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.769135 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.769136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.769283 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.769845 (+ 562us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:14:45.769845 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:45.769878 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:14:45.769905 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:45.770109 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:45.770114 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:45.770118 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:14:45.770173 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 23:14:45.770300 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260924 23:14:45.771050 665 heartbeater.cc:347] Connected to a master server at 127.12.212.62:39475
I20260924 23:14:45.771145 665 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:45.771370 665 heartbeater.cc:511] Master 127.12.212.62:39475 requested a full tablet report, sending...
I20260924 23:14:45.771790 346 ts_manager.cc:194] Registered new tserver with Master: 0f442631ba9443638f45397461017cd1 (127.12.212.2:46533)
I20260924 23:14:45.772171 346 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.212.2:34627
I20260924 23:14:45.772847 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 537
I20260924 23:14:45.772943 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-1/wal/instance
I20260924 23:14:45.773166 13136 external_mini_cluster.cc:1428] Setting key 0368b099401e2e53f2f2fd0c9a256d95
I20260924 23:14:45.774034 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.212.3:0
--local_ip_for_outbound_sockets=127.12.212.3
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:39475
--builtin_ntp_servers=127.12.212.20:41421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 23:14:45.854449 668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:45.854683 668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:45.854733 668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:45.855993 668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:45.856045 668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:45.856071 668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.212.3
I20260924 23:14:45.857466 668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:41421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.212.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.212.3
--webserver_port=0
--tserver_master_addrs=127.12.212.62:39475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.212.3
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:45.857723 668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:45.858022 668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:45.862967 674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:45.862962 673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:45.863072 676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:45.863222 668 server_base.cc:1034] running on GCE node
I20260924 23:14:45.863483 668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:45.863716 668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:45.864881 668 hybrid_clock.cc:648] HybridClock initialized: now 1790291685864868 us; error 35 us; skew 500 ppm
I20260924 23:14:45.866222 668 webserver.cc:460] Webserver started at http://127.12.212.3:34995/ using document root <none> and password file <none>
I20260924 23:14:45.866487 668 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:45.866533 668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:45.866653 668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:45.867852 668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-2/data/instance:
uuid: "f4f3be1545a141c7b8c4d553e23eff2a"
format_stamp: "Formatted at 2026-09-24 23:14:45 on dist-test-slave-m0z6"
server_key: "52368af35acca93780e738d5e919087b"
server_key_iv: "4236a313fbdb3e8c4677aa1e875ee929"
server_key_version: "encryptionkey@0"
I20260924 23:14:45.868220 668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance:
uuid: "f4f3be1545a141c7b8c4d553e23eff2a"
format_stamp: "Formatted at 2026-09-24 23:14:45 on dist-test-slave-m0z6"
server_key: "52368af35acca93780e738d5e919087b"
server_key_iv: "4236a313fbdb3e8c4677aa1e875ee929"
server_key_version: "encryptionkey@0"
I20260924 23:14:45.869534 668 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:14:45.870473 682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:45.870679 668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:45.870790 668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-2/wal
uuid: "f4f3be1545a141c7b8c4d553e23eff2a"
format_stamp: "Formatted at 2026-09-24 23:14:45 on dist-test-slave-m0z6"
server_key: "52368af35acca93780e738d5e919087b"
server_key_iv: "4236a313fbdb3e8c4677aa1e875ee929"
server_key_version: "encryptionkey@0"
I20260924 23:14:45.870850 668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:45.902123 668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:45.902421 668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:45.902531 668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:45.902822 668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 23:14:45.903134 668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 23:14:45.903167 668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:45.903190 668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 23:14:45.903203 668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:45.908699 668 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.3:40327
I20260924 23:14:45.908735 795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.3:40327 every 8 connection(s)
I20260924 23:14:45.909102 668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-2/data/info.pb
I20260924 23:14:45.914573 396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.909796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.3:44533 (local address 127.12.212.62:39475)
0924 23:14:45.909846 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:45.909848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:45.910048 (+ 200us) server_negotiation.cc:407] Connection header received
0924 23:14:45.910632 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:45.910634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:45.910747 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:45.910808 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:45.911506 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.912011 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.913026 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.913274 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.913350 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.914321 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:45.914322 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:45.914323 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:45.914327 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:45.914345 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:45.914349 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:45.914400 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:45.914466 (+ 66us) server_negotiation.cc:299] Negotiation successful
0924 23:14:45.914506 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 23:14:45.914790 798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.909782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:39475 (local address 127.12.212.3:44533)
0924 23:14:45.909943 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:45.909977 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:45.910468 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:45.910877 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:45.910881 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:45.910890 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:14:45.911416 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.911421 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.912087 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.912088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.912933 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.912936 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.913390 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.913391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.913539 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.914167 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 23:14:45.914167 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:14:45.914200 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 23:14:45.914225 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:14:45.914407 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:14:45.914410 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:14:45.914412 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 23:14:45.914461 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 23:14:45.914556 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
I20260924 23:14:45.915341 796 heartbeater.cc:347] Connected to a master server at 127.12.212.62:39475
I20260924 23:14:45.915441 796 heartbeater.cc:464] Registering TS with master...
I20260924 23:14:45.915660 796 heartbeater.cc:511] Master 127.12.212.62:39475 requested a full tablet report, sending...
I20260924 23:14:45.916019 346 ts_manager.cc:194] Registered new tserver with Master: f4f3be1545a141c7b8c4d553e23eff2a (127.12.212.3:40327)
I20260924 23:14:45.916414 346 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.212.3:44533
I20260924 23:14:45.918951 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 668
I20260924 23:14:45.919049 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-2/wal/instance
I20260924 23:14:45.919214 13136 external_mini_cluster.cc:1428] Setting key 781ca0d970e6831daacd12ffc3332251
I20260924 23:14:45.920081 13136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 23:14:45.924490 396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.921428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39592 (local address 127.12.212.62:39475)
0924 23:14:45.921485 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:45.921487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:45.921601 (+ 114us) server_negotiation.cc:407] Connection header received
0924 23:14:45.921670 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:45.921671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:45.921681 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:45.921730 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:45.921889 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.922402 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.923280 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.923472 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.923534 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.924251 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:45.924252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:45.924253 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:45.924257 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:45.924273 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:45.924277 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:45.924339 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:45.924391 (+ 52us) server_negotiation.cc:299] Negotiation successful
0924 23:14:45.924419 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 23:14:45.926388 346 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 23:14:45.928102 346 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 23:14:45.934916 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.930540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43530 (local address 127.12.212.1:46311)
0924 23:14:45.930774 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:45.930778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:45.930786 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:14:45.930826 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:45.930828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:45.930844 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:45.930919 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:45.931167 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.932421 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.933650 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.934439 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.934526 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.934620 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:45.934777 (+ 157us) server_negotiation.cc:299] Negotiation successful
0924 23:14:45.934822 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":111,"threads_started":1}
I20260924 23:14:45.934928 805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.930409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.2:46533 (local address 127.0.0.1:37626)
0924 23:14:45.930863 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:45.930874 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:45.930946 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:45.931215 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:45.931216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:45.931218 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:45.931263 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.931264 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.932605 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.932606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.933764 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.933767 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.934626 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.934627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.934677 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.934680 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:14:45.934743 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 23:14:45.934822 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":53,"thread_start_us":76,"threads_started":1}
I20260924 23:14:45.934935 804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.930369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:46311 (local address 127.0.0.1:43530)
0924 23:14:45.930490 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:45.930534 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:45.930605 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:45.931005 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:45.931009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:45.931011 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:45.931082 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.931083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.932487 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.932488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.933557 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.933560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.934526 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.934527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.934615 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.934631 (+ 16us) client_negotiation.cc:769] Sending connection context
0924 23:14:45.934754 (+ 123us) client_negotiation.cc:240] Negotiation successful
0924 23:14:45.934822 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":123,"thread_start_us":69,"threads_started":1}
I20260924 23:14:45.935298 806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.930535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37626 (local address 127.12.212.2:46533)
0924 23:14:45.931054 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:45.931059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:45.931069 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:14:45.931106 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:45.931108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:45.931123 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:45.931204 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:45.931325 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.932416 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.933855 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.934426 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.934494 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.934595 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:45.935168 (+ 573us) server_negotiation.cc:299] Negotiation successful
0924 23:14:45.935219 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":86,"threads_started":1}
I20260924 23:14:45.935431 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.930578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:40327 (local address 127.0.0.1:47158)
0924 23:14:45.930866 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:45.930874 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:45.930931 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:45.931399 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:45.931400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:45.931401 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:45.931426 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.931427 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.932414 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.932415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.933869 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:45.933871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.934883 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.934884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:45.934948 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.934952 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:14:45.934999 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 23:14:45.935362 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":43,"thread_start_us":67,"threads_started":1}
I20260924 23:14:45.935726 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:45.930854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47158 (local address 127.12.212.3:40327)
0924 23:14:45.931199 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:45.931204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:45.931213 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:14:45.931249 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:45.931251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:45.931266 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:45.931337 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:45.931487 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.932342 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.933945 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:45.934682 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:45.934768 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:45.934849 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:45.935586 (+ 737us) server_negotiation.cc:299] Negotiation successful
0924 23:14:45.935645 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":84,"threads_started":1}
I20260924 23:14:45.936008 599 tablet_service.cc:1467] Processing CreateTablet for tablet 3cb3ea3c33844474b7b1a2874426c8b3 (DEFAULT_TABLE table=test-table [id=c449e24826014b23a481547ac3ade813]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:45.935956 468 tablet_service.cc:1467] Processing CreateTablet for tablet 3cb3ea3c33844474b7b1a2874426c8b3 (DEFAULT_TABLE table=test-table [id=c449e24826014b23a481547ac3ade813]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:45.936332 599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cb3ea3c33844474b7b1a2874426c8b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:45.936332 468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cb3ea3c33844474b7b1a2874426c8b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:45.936311 730 tablet_service.cc:1467] Processing CreateTablet for tablet 3cb3ea3c33844474b7b1a2874426c8b3 (DEFAULT_TABLE table=test-table [id=c449e24826014b23a481547ac3ade813]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 23:14:45.936568 730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cb3ea3c33844474b7b1a2874426c8b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:45.939188 810 tablet_bootstrap.cc:492] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a: Bootstrap starting.
I20260924 23:14:45.939721 811 tablet_bootstrap.cc:492] T 3cb3ea3c33844474b7b1a2874426c8b3 P 62b8ff0d48c242c19f12d6d6a383f9c1: Bootstrap starting.
I20260924 23:14:45.939836 812 tablet_bootstrap.cc:492] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1: Bootstrap starting.
I20260924 23:14:45.940069 810 tablet_bootstrap.cc:654] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:45.940380 811 tablet_bootstrap.cc:654] T 3cb3ea3c33844474b7b1a2874426c8b3 P 62b8ff0d48c242c19f12d6d6a383f9c1: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:45.940387 810 log.cc:824] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:45.940410 812 tablet_bootstrap.cc:654] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:45.940691 812 log.cc:824] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:45.940728 811 log.cc:824] T 3cb3ea3c33844474b7b1a2874426c8b3 P 62b8ff0d48c242c19f12d6d6a383f9c1: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:45.941291 810 tablet_bootstrap.cc:492] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a: No bootstrap required, opened a new log
I20260924 23:14:45.941344 810 ts_tablet_manager.cc:1397] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:14:45.941409 812 tablet_bootstrap.cc:492] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1: No bootstrap required, opened a new log
I20260924 23:14:45.941439 811 tablet_bootstrap.cc:492] T 3cb3ea3c33844474b7b1a2874426c8b3 P 62b8ff0d48c242c19f12d6d6a383f9c1: No bootstrap required, opened a new log
I20260924 23:14:45.941489 811 ts_tablet_manager.cc:1397] T 3cb3ea3c33844474b7b1a2874426c8b3 P 62b8ff0d48c242c19f12d6d6a383f9c1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:14:45.941490 812 ts_tablet_manager.cc:1397] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 23:14:45.943497 810 raft_consensus.cc:348] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f442631ba9443638f45397461017cd1" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46533 } } peers { permanent_uuid: "62b8ff0d48c242c19f12d6d6a383f9c1" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46311 } } peers { permanent_uuid: "f4f3be1545a141c7b8c4d553e23eff2a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40327 } }
I20260924 23:14:45.943631 810 raft_consensus.cc:374] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:45.943652 810 raft_consensus.cc:729] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4f3be1545a141c7b8c4d553e23eff2a, State: Initialized, Role: FOLLOWER
I20260924 23:14:45.943750 810 consensus_queue.cc:260] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f442631ba9443638f45397461017cd1" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46533 } } peers { permanent_uuid: "62b8ff0d48c242c19f12d6d6a383f9c1" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46311 } } peers { permanent_uuid: "f4f3be1545a141c7b8c4d553e23eff2a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40327 } }
I20260924 23:14:45.943980 810 ts_tablet_manager.cc:1428] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:14:45.943986 811 raft_consensus.cc:348] T 3cb3ea3c33844474b7b1a2874426c8b3 P 62b8ff0d48c242c19f12d6d6a383f9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f442631ba9443638f45397461017cd1" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46533 } } peers { permanent_uuid: "62b8ff0d48c242c19f12d6d6a383f9c1" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46311 } } peers { permanent_uuid: "f4f3be1545a141c7b8c4d553e23eff2a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40327 } }
I20260924 23:14:45.943986 812 raft_consensus.cc:348] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f442631ba9443638f45397461017cd1" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46533 } } peers { permanent_uuid: "62b8ff0d48c242c19f12d6d6a383f9c1" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46311 } } peers { permanent_uuid: "f4f3be1545a141c7b8c4d553e23eff2a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40327 } }
I20260924 23:14:45.944139 812 raft_consensus.cc:374] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:45.944137 811 raft_consensus.cc:374] T 3cb3ea3c33844474b7b1a2874426c8b3 P 62b8ff0d48c242c19f12d6d6a383f9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:45.944171 812 raft_consensus.cc:729] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f442631ba9443638f45397461017cd1, State: Initialized, Role: FOLLOWER
I20260924 23:14:45.944180 811 raft_consensus.cc:729] T 3cb3ea3c33844474b7b1a2874426c8b3 P 62b8ff0d48c242c19f12d6d6a383f9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62b8ff0d48c242c19f12d6d6a383f9c1, State: Initialized, Role: FOLLOWER
I20260924 23:14:45.944279 811 consensus_queue.cc:260] T 3cb3ea3c33844474b7b1a2874426c8b3 P 62b8ff0d48c242c19f12d6d6a383f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f442631ba9443638f45397461017cd1" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46533 } } peers { permanent_uuid: "62b8ff0d48c242c19f12d6d6a383f9c1" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46311 } } peers { permanent_uuid: "f4f3be1545a141c7b8c4d553e23eff2a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40327 } }
I20260924 23:14:45.944279 812 consensus_queue.cc:260] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f442631ba9443638f45397461017cd1" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46533 } } peers { permanent_uuid: "62b8ff0d48c242c19f12d6d6a383f9c1" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46311 } } peers { permanent_uuid: "f4f3be1545a141c7b8c4d553e23eff2a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40327 } }
I20260924 23:14:45.944519 812 ts_tablet_manager.cc:1428] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:14:45.944519 811 ts_tablet_manager.cc:1428] T 3cb3ea3c33844474b7b1a2874426c8b3 P 62b8ff0d48c242c19f12d6d6a383f9c1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 23:14:45.944568 665 heartbeater.cc:503] Master 127.12.212.62:39475 was elected leader, sending a full tablet report...
I20260924 23:14:45.944631 534 heartbeater.cc:503] Master 127.12.212.62:39475 was elected leader, sending a full tablet report...
I20260924 23:14:45.944763 796 heartbeater.cc:503] Master 127.12.212.62:39475 was elected leader, sending a full tablet report...
W20260924 23:14:46.015219 666 tablet.cc:2369] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 23:14:46.068511 818 raft_consensus.cc:482] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 23:14:46.068655 818 raft_consensus.cc:504] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f442631ba9443638f45397461017cd1" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46533 } } peers { permanent_uuid: "62b8ff0d48c242c19f12d6d6a383f9c1" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46311 } } peers { permanent_uuid: "f4f3be1545a141c7b8c4d553e23eff2a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40327 } }
I20260924 23:14:46.069036 818 leader_election.cc:288] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62b8ff0d48c242c19f12d6d6a383f9c1 (127.12.212.1:46311), f4f3be1545a141c7b8c4d553e23eff2a (127.12.212.3:40327)
I20260924 23:14:46.071687 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:46.069294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:43645 (local address 127.12.212.3:40327)
0924 23:14:46.069357 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:46.069359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:46.069371 (+ 12us) server_negotiation.cc:407] Connection header received
0924 23:14:46.069404 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:46.069405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:46.069413 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:46.069463 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:46.069628 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:46.070085 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:46.071031 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:46.071386 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:46.071448 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:46.071457 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:46.071584 (+ 127us) server_negotiation.cc:299] Negotiation successful
0924 23:14:46.071621 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 23:14:46.071687 667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:46.069238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.3:40327 (local address 127.12.212.2:43645)
0924 23:14:46.069279 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:46.069291 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:46.069333 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:46.069486 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:46.069487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:46.069489 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:46.069552 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:46.069554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:46.070161 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:46.070162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:46.070952 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:46.070954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:46.071454 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:46.071454 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:46.071518 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:46.071522 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 23:14:46.071564 (+ 42us) client_negotiation.cc:240] Negotiation successful
0924 23:14:46.071609 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260924 23:14:46.071952 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:46.069326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.212.2:49459 (local address 127.12.212.1:46311)
0924 23:14:46.069385 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:46.069387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:46.069588 (+ 201us) server_negotiation.cc:407] Connection header received
0924 23:14:46.069659 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:46.069660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:46.069671 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:46.069717 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 23:14:46.069852 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:46.070304 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:46.071242 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:46.071616 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:46.071676 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:46.071683 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:46.071839 (+ 156us) server_negotiation.cc:299] Negotiation successful
0924 23:14:46.071885 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 23:14:46.072052 750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cb3ea3c33844474b7b1a2874426c8b3" candidate_uuid: "0f442631ba9443638f45397461017cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4f3be1545a141c7b8c4d553e23eff2a" is_pre_election: true
I20260924 23:14:46.072176 819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:46.069239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:46311 (local address 127.12.212.2:49459)
0924 23:14:46.069529 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:46.069538 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:14:46.069585 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:14:46.069724 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:14:46.069725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:14:46.069726 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 23:14:46.069796 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:46.069798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:46.070374 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:46.070375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:46.071164 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:14:46.071166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:14:46.071687 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:14:46.071688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:14:46.071752 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:46.071755 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 23:14:46.072046 (+ 291us) client_negotiation.cc:240] Negotiation successful
0924 23:14:46.072095 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":138,"thread_start_us":89,"threads_started":1}
I20260924 23:14:46.072216 750 raft_consensus.cc:2454] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f442631ba9443638f45397461017cd1 in term 0.
I20260924 23:14:46.072446 553 leader_election.cc:302] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f442631ba9443638f45397461017cd1, f4f3be1545a141c7b8c4d553e23eff2a; no voters:
I20260924 23:14:46.072479 488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cb3ea3c33844474b7b1a2874426c8b3" candidate_uuid: "0f442631ba9443638f45397461017cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62b8ff0d48c242c19f12d6d6a383f9c1" is_pre_election: true
I20260924 23:14:46.072569 818 raft_consensus.cc:2781] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 23:14:46.072618 818 raft_consensus.cc:482] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 23:14:46.072640 818 raft_consensus.cc:3037] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:46.072666 488 raft_consensus.cc:2454] T 3cb3ea3c33844474b7b1a2874426c8b3 P 62b8ff0d48c242c19f12d6d6a383f9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f442631ba9443638f45397461017cd1 in term 0.
I20260924 23:14:46.073433 818 raft_consensus.cc:504] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f442631ba9443638f45397461017cd1" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46533 } } peers { permanent_uuid: "62b8ff0d48c242c19f12d6d6a383f9c1" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46311 } } peers { permanent_uuid: "f4f3be1545a141c7b8c4d553e23eff2a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40327 } }
I20260924 23:14:46.073563 818 leader_election.cc:288] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 62b8ff0d48c242c19f12d6d6a383f9c1 (127.12.212.1:46311), f4f3be1545a141c7b8c4d553e23eff2a (127.12.212.3:40327)
I20260924 23:14:46.073727 750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cb3ea3c33844474b7b1a2874426c8b3" candidate_uuid: "0f442631ba9443638f45397461017cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4f3be1545a141c7b8c4d553e23eff2a"
I20260924 23:14:46.073737 488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cb3ea3c33844474b7b1a2874426c8b3" candidate_uuid: "0f442631ba9443638f45397461017cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62b8ff0d48c242c19f12d6d6a383f9c1"
I20260924 23:14:46.073807 750 raft_consensus.cc:3037] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:46.073810 488 raft_consensus.cc:3037] T 3cb3ea3c33844474b7b1a2874426c8b3 P 62b8ff0d48c242c19f12d6d6a383f9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:46.074523 750 raft_consensus.cc:2454] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f442631ba9443638f45397461017cd1 in term 1.
I20260924 23:14:46.074558 488 raft_consensus.cc:2454] T 3cb3ea3c33844474b7b1a2874426c8b3 P 62b8ff0d48c242c19f12d6d6a383f9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f442631ba9443638f45397461017cd1 in term 1.
I20260924 23:14:46.074715 553 leader_election.cc:302] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f442631ba9443638f45397461017cd1, f4f3be1545a141c7b8c4d553e23eff2a; no voters:
I20260924 23:14:46.074818 818 raft_consensus.cc:2781] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:46.074988 818 raft_consensus.cc:686] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 0f442631ba9443638f45397461017cd1, State: Running, Role: LEADER
I20260924 23:14:46.075079 818 consensus_queue.cc:237] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f442631ba9443638f45397461017cd1" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46533 } } peers { permanent_uuid: "62b8ff0d48c242c19f12d6d6a383f9c1" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46311 } } peers { permanent_uuid: "f4f3be1545a141c7b8c4d553e23eff2a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40327 } }
I20260924 23:14:46.075946 346 catalog_manager.cc:5495] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f442631ba9443638f45397461017cd1 (127.12.212.2). New cstate: current_term: 1 leader_uuid: "0f442631ba9443638f45397461017cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f442631ba9443638f45397461017cd1" member_type: VOTER last_known_addr { host: "127.12.212.2" port: 46533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62b8ff0d48c242c19f12d6d6a383f9c1" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4f3be1545a141c7b8c4d553e23eff2a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40327 } health_report { overall_health: UNKNOWN } } }
I20260924 23:14:46.082787 806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:46.080444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37642 (local address 127.12.212.2:46533)
0924 23:14:46.080500 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:46.080502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:46.080510 (+ 8us) server_negotiation.cc:407] Connection header received
0924 23:14:46.080545 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:46.080546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:46.080574 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:46.080656 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:14:46.080798 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:46.081291 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:46.082021 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:46.082229 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:46.082284 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:46.082432 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:14:46.082555 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:14:46.082635 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:46.082652 (+ 17us) server_negotiation.cc:299] Negotiation successful
0924 23:14:46.082672 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 23:14:46.084467 750 raft_consensus.cc:1261] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a [term 1 FOLLOWER]: Refusing update from remote peer 0f442631ba9443638f45397461017cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:14:46.084483 488 raft_consensus.cc:1261] T 3cb3ea3c33844474b7b1a2874426c8b3 P 62b8ff0d48c242c19f12d6d6a383f9c1 [term 1 FOLLOWER]: Refusing update from remote peer 0f442631ba9443638f45397461017cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:14:46.084823 820 consensus_queue.cc:1035] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4f3be1545a141c7b8c4d553e23eff2a" member_type: VOTER last_known_addr { host: "127.12.212.3" port: 40327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:14:46.084939 818 consensus_queue.cc:1035] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62b8ff0d48c242c19f12d6d6a383f9c1" member_type: VOTER last_known_addr { host: "127.12.212.1" port: 46311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:14:46.087239 823 mvcc.cc:205] Tried to move back new op lower bound from 7333034746199572480 to 7333034746164117504. Current Snapshot: MvccSnapshot[applied={T|T < 7333034746199572480}]
I20260924 23:14:46.090152 345 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:14:46.090269 345 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 23:14:46.091151 345 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb9ec23cfc34441bad52bc824cda814d: Sending DeleteTablet for 3 replicas of tablet 3cb3ea3c33844474b7b1a2874426c8b3
I20260924 23:14:46.091511 599 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cb3ea3c33844474b7b1a2874426c8b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:46 UTC) from {username='slave'} at 127.0.0.1:37626
I20260924 23:14:46.091591 730 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cb3ea3c33844474b7b1a2874426c8b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:46 UTC) from {username='slave'} at 127.0.0.1:47158
I20260924 23:14:46.091835 833 tablet_replica.cc:330] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a: stopping tablet replica
I20260924 23:14:46.091823 468 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cb3ea3c33844474b7b1a2874426c8b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 23:14:46 UTC) from {username='slave'} at 127.0.0.1:43530
I20260924 23:14:46.091979 833 raft_consensus.cc:2229] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 23:14:46.092042 832 tablet_replica.cc:330] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1: stopping tablet replica
I20260924 23:14:46.092141 832 raft_consensus.cc:2229] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [term 1 LEADER]: Raft consensus shutting down.
I20260924 23:14:46.092141 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 406
I20260924 23:14:46.092272 832 raft_consensus.cc:2258] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:14:46.093019 832 ts_tablet_manager.cc:1905] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:14:46.092293 833 raft_consensus.cc:2258] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 23:14:46.094445 833 ts_tablet_manager.cc:1905] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 23:14:46.094849 832 ts_tablet_manager.cc:1918] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:14:46.094931 832 log.cc:1196] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-1/wal/wals/3cb3ea3c33844474b7b1a2874426c8b3
I20260924 23:14:46.095185 832 ts_tablet_manager.cc:1939] T 3cb3ea3c33844474b7b1a2874426c8b3 P 0f442631ba9443638f45397461017cd1: Deleting consensus metadata
I20260924 23:14:46.095625 331 catalog_manager.cc:4841] TS 0f442631ba9443638f45397461017cd1 (127.12.212.2:46533): tablet 3cb3ea3c33844474b7b1a2874426c8b3 (table test-table [id=c449e24826014b23a481547ac3ade813]) successfully deleted
I20260924 23:14:46.096354 833 ts_tablet_manager.cc:1918] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 23:14:46.096431 833 log.cc:1196] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a: Deleting WAL directory at /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790291444018253-13136-0/minicluster-data/ts-2/wal/wals/3cb3ea3c33844474b7b1a2874426c8b3
I20260924 23:14:46.096653 833 ts_tablet_manager.cc:1939] T 3cb3ea3c33844474b7b1a2874426c8b3 P f4f3be1545a141c7b8c4d553e23eff2a: Deleting consensus metadata
I20260924 23:14:46.097077 331 catalog_manager.cc:4841] TS f4f3be1545a141c7b8c4d553e23eff2a (127.12.212.3:40327): tablet 3cb3ea3c33844474b7b1a2874426c8b3 (table test-table [id=c449e24826014b23a481547ac3ade813]) successfully deleted
W20260924 23:14:46.098030 332 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.212.1:46311 (error 108)
I20260924 23:14:46.098246 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 537
I20260924 23:14:46.098946 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:46.098824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.1:46311 (local address 127.0.0.1:43542)
0924 23:14:46.098872 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 23:14:46.098911 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.1:46311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 23:14:46.099166 332 catalog_manager.cc:4584] TS 62b8ff0d48c242c19f12d6d6a383f9c1 (127.12.212.1:46311): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3cb3ea3c33844474b7b1a2874426c8b3: Network error: Client connection negotiation failed: client connection to 127.12.212.1:46311: connect: Connection refused (error 111)
I20260924 23:14:46.102149 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 668
I20260924 23:14:46.106190 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 315
2026-09-24T23:14:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (826 ms)
[----------] 6 tests from EncryptionPolicyTest (8262 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:14:46 dist-test-slave-m0z6 krb5kdc[838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:14:46 dist-test-slave-m0z6 krb5kdc[838](info): set up 2 sockets
Sep 24 23:14:46 dist-test-slave-m0z6 krb5kdc[838](info): commencing operation
krb5kdc: starting...
W20260924 23:14:47.062498 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:14:47 dist-test-slave-m0z6 krb5kdc[838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:14:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:14:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 23:14:47.177935 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38359
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:47.258108 851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:47.258322 851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:47.258342 851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:47.259524 851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:14:47.259565 851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:47.259578 851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:47.259589 851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:14:47.259600 851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:14:47.261026 851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38359
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:47.261263 851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:47.261471 851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:47.266075 856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:47.266064 857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:47.266076 859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:47.266360 851 server_base.cc:1034] running on GCE node
I20260924 23:14:47.266594 851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:47.266911 851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:47.268054 851 hybrid_clock.cc:648] HybridClock initialized: now 1790291687268036 us; error 33 us; skew 500 ppm
Sep 24 23:14:47 dist-test-slave-m0z6 krb5kdc[838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291687, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:47.270421 851 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 23:14:47.270977 851 webserver.cc:460] Webserver started at http://127.0.0.1:40893/ using document root <none> and password file <none>
I20260924 23:14:47.271190 851 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:47.271235 851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:47.271343 851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:47.272383 851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "c0779788b2944944a722242e1645c327"
format_stamp: "Formatted at 2026-09-24 23:14:47 on dist-test-slave-m0z6"
server_key: "8e35fa8da00a61e43faf8982585d36f9"
server_key_iv: "fe093b55d3168fa4563d38d3034d4b4b"
server_key_version: "encryptionkey@0"
I20260924 23:14:47.272716 851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "c0779788b2944944a722242e1645c327"
format_stamp: "Formatted at 2026-09-24 23:14:47 on dist-test-slave-m0z6"
server_key: "8e35fa8da00a61e43faf8982585d36f9"
server_key_iv: "fe093b55d3168fa4563d38d3034d4b4b"
server_key_version: "encryptionkey@0"
I20260924 23:14:47.274014 851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:14:47.274729 866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:47.274883 851 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:14:47.274966 851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "c0779788b2944944a722242e1645c327"
format_stamp: "Formatted at 2026-09-24 23:14:47 on dist-test-slave-m0z6"
server_key: "8e35fa8da00a61e43faf8982585d36f9"
server_key_iv: "fe093b55d3168fa4563d38d3034d4b4b"
server_key_version: "encryptionkey@0"
I20260924 23:14:47.275033 851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:47.286303 851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:47.288282 851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:47.288439 851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:47.292333 851 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38359
I20260924 23:14:47.292374 918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38359 every 8 connection(s)
I20260924 23:14:47.292719 851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:14:47.292917 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 851
I20260924 23:14:47.293046 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:14:47.293242 13136 external_mini_cluster.cc:1428] Setting key a41fd0a78a204bce1585a3a872771cd3
I20260924 23:14:47.293618 919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:47.297025 919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327: Bootstrap starting.
I20260924 23:14:47.297667 919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:47.297958 919 log.cc:824] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:47.298745 919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327: No bootstrap required, opened a new log
Sep 24 23:14:47 dist-test-slave-m0z6 krb5kdc[838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 23:14:47.300912 919 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0779788b2944944a722242e1645c327" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38359 } }
I20260924 23:14:47.301079 919 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:47.301115 919 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0779788b2944944a722242e1645c327, State: Initialized, Role: FOLLOWER
I20260924 23:14:47.301256 919 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0779788b2944944a722242e1645c327" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38359 } }
I20260924 23:14:47.301342 919 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:14:47.301378 919 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:14:47.301419 919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:47.302106 919 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0779788b2944944a722242e1645c327" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38359 } }
I20260924 23:14:47.302210 919 leader_election.cc:302] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0779788b2944944a722242e1645c327; no voters:
I20260924 23:14:47.302392 919 leader_election.cc:288] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:14:47.302438 924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:47.302551 924 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327 [term 1 LEADER]: Becoming Leader. State: Replica: c0779788b2944944a722242e1645c327, State: Running, Role: LEADER
I20260924 23:14:47.302635 924 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0779788b2944944a722242e1645c327" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38359 } }
I20260924 23:14:47.302680 919 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:14:47.303045 925 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0779788b2944944a722242e1645c327" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0779788b2944944a722242e1645c327" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38359 } } }
I20260924 23:14:47.303054 926 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0779788b2944944a722242e1645c327. Latest consensus state: current_term: 1 leader_uuid: "c0779788b2944944a722242e1645c327" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0779788b2944944a722242e1645c327" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38359 } } }
I20260924 23:14:47.303134 925 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:47.303150 926 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:47.303622 929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:14:47.304211 929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:14:47.304772 922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:47.293755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38801 (local address 127.0.0.1:38359)
0924 23:14:47.293912 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:47.293922 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:47.293954 (+ 32us) server_negotiation.cc:407] Connection header received
0924 23:14:47.294493 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:47.294510 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:47.294588 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:47.294830 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:47.295421 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:47.296776 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:47.297603 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:47.298220 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:47.298325 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:47.300929 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:47.300934 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:47.300934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:47.300948 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:47.302846 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:47.303353 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:47.303356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:47.303356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:47.303460 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:47.303723 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:47.303725 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:47.303725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:47.304094 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:47.304248 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:47.304437 (+ 189us) server_negotiation.cc:299] Negotiation successful
0924 23:14:47.304561 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260924 23:14:47.305955 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:14:47.306098 929 catalog_manager.cc:1347] Generated new cluster ID: d1c27a3e80e94261bc340e4e15f2eea5
I20260924 23:14:47.306147 929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:14:47.314481 922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:47.307456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55755 (local address 127.0.0.1:38359)
0924 23:14:47.307491 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:47.307493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:47.307592 (+ 99us) server_negotiation.cc:407] Connection header received
0924 23:14:47.307637 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:47.307638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:47.307648 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:47.307715 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:47.308290 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:47.309059 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:47.310013 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:47.310346 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:47.310421 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:47.311779 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:47.311781 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:47.311782 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:47.311791 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:47.313304 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:47.313694 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:47.313695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:47.313695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:47.313757 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:47.313976 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:47.313977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:47.313978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:47.314111 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:47.314215 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:47.314323 (+ 108us) server_negotiation.cc:299] Negotiation successful
0924 23:14:47.314386 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 23:14:47.318598 929 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:14:47.319388 929 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:14:47.325845 929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0779788b2944944a722242e1645c327: Generated new TSK 0
I20260924 23:14:47.326059 929 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:14:47.332647 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 851
W20260924 23:14:47.336418 836 connection.cc:535] client connection to 127.0.0.1:38359 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38359): Connection reset by peer (error 104)
2026-09-24T23:14:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1224 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:14:47 dist-test-slave-m0z6 krb5kdc[950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:14:47 dist-test-slave-m0z6 krb5kdc[950](info): set up 2 sockets
Sep 24 23:14:47 dist-test-slave-m0z6 krb5kdc[950](info): commencing operation
krb5kdc: starting...
W20260924 23:14:48.290570 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:14:48 dist-test-slave-m0z6 krb5kdc[950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:14:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:14:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 23:14:48.409762 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32971
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:48.493350 963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:48.493590 963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:48.493625 963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:48.494995 963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:14:48.495049 963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:48.495069 963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:48.495085 963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:14:48.495108 963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:14:48.496773 963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32971
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:48.497042 963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:48.497272 963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:48.502013 971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:48.502025 969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:48.502066 968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:48.502408 963 server_base.cc:1034] running on GCE node
I20260924 23:14:48.502647 963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:48.502951 963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:48.504122 963 hybrid_clock.cc:648] HybridClock initialized: now 1790291688504104 us; error 31 us; skew 500 ppm
Sep 24 23:14:48 dist-test-slave-m0z6 krb5kdc[950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291688, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:48.512688 963 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 23:14:48.513196 963 webserver.cc:460] Webserver started at http://127.0.0.1:39665/ using document root <none> and password file <none>
I20260924 23:14:48.513434 963 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:48.513484 963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:48.513620 963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:48.514796 963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "fd480c771cef4034a0ecadf7d48acf4d"
format_stamp: "Formatted at 2026-09-24 23:14:48 on dist-test-slave-m0z6"
server_key: "d297d3e864c9ae3343f97f4d980c3e8f"
server_key_iv: "bad6949a4af8da40a204bc688f44d400"
server_key_version: "encryptionkey@0"
I20260924 23:14:48.515142 963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "fd480c771cef4034a0ecadf7d48acf4d"
format_stamp: "Formatted at 2026-09-24 23:14:48 on dist-test-slave-m0z6"
server_key: "d297d3e864c9ae3343f97f4d980c3e8f"
server_key_iv: "bad6949a4af8da40a204bc688f44d400"
server_key_version: "encryptionkey@0"
I20260924 23:14:48.516503 963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:14:48.517292 978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:48.517462 963 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:14:48.517546 963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "fd480c771cef4034a0ecadf7d48acf4d"
format_stamp: "Formatted at 2026-09-24 23:14:48 on dist-test-slave-m0z6"
server_key: "d297d3e864c9ae3343f97f4d980c3e8f"
server_key_iv: "bad6949a4af8da40a204bc688f44d400"
server_key_version: "encryptionkey@0"
I20260924 23:14:48.517604 963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:48.532473 963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:48.534847 963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:48.534976 963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:48.538795 963 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32971
I20260924 23:14:48.538828 1030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32971 every 8 connection(s)
I20260924 23:14:48.539194 963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:14:48.540068 1031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:48.542373 1031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d: Bootstrap starting.
I20260924 23:14:48.542977 1031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:48.543231 1031 log.cc:824] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:48.543953 1031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d: No bootstrap required, opened a new log
I20260924 23:14:48.544840 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 963
I20260924 23:14:48.544938 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:14:48.545153 13136 external_mini_cluster.cc:1428] Setting key f8bdf9c24ee3841969d35567b22614a5
I20260924 23:14:48.545749 1031 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd480c771cef4034a0ecadf7d48acf4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32971 } }
I20260924 23:14:48.545917 1031 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:48.545951 1031 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd480c771cef4034a0ecadf7d48acf4d, State: Initialized, Role: FOLLOWER
I20260924 23:14:48.546083 1031 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd480c771cef4034a0ecadf7d48acf4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32971 } }
I20260924 23:14:48.546165 1031 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:14:48.546196 1031 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:14:48.546243 1031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:48.546926 1031 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd480c771cef4034a0ecadf7d48acf4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32971 } }
I20260924 23:14:48.547019 1031 leader_election.cc:302] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd480c771cef4034a0ecadf7d48acf4d; no voters:
I20260924 23:14:48.547152 1031 leader_election.cc:288] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:14:48.547215 1036 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:48.547392 1036 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d [term 1 LEADER]: Becoming Leader. State: Replica: fd480c771cef4034a0ecadf7d48acf4d, State: Running, Role: LEADER
I20260924 23:14:48.547422 1031 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:14:48.547487 1036 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd480c771cef4034a0ecadf7d48acf4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32971 } }
I20260924 23:14:48.547842 1036 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd480c771cef4034a0ecadf7d48acf4d. Latest consensus state: current_term: 1 leader_uuid: "fd480c771cef4034a0ecadf7d48acf4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd480c771cef4034a0ecadf7d48acf4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32971 } } }
I20260924 23:14:48.547904 1036 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:48.547888 1037 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd480c771cef4034a0ecadf7d48acf4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd480c771cef4034a0ecadf7d48acf4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32971 } } }
I20260924 23:14:48.547994 1037 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d [sys.catalog]: This master's current role is: LEADER
W20260924 23:14:48.549280 1052 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:14:48.549383 1052 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:14:48.549437 1043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:14:48.549588 1043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:14:48.551330 1043 catalog_manager.cc:1347] Generated new cluster ID: 917d703415fc4a82b0d07366e5741aa9
I20260924 23:14:48.551378 1043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:14:48.551397 1035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:48.545676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49783 (local address 127.0.0.1:32971)
0924 23:14:48.545850 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:48.545859 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:48.545884 (+ 25us) server_negotiation.cc:407] Connection header received
0924 23:14:48.545993 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:48.546001 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:48.546063 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:48.546204 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:48.546431 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:48.548175 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:48.549241 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:48.549835 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:48.549940 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:48.550657 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:48.550661 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:48.550661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:48.550889 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:48.550934 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:48.550950 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:48.551029 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:48.551132 (+ 103us) server_negotiation.cc:299] Negotiation successful
0924 23:14:48.551161 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260924 23:14:48.552335 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:14:48.557826 1035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:48.553880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34727 (local address 127.0.0.1:32971)
0924 23:14:48.553944 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:48.553946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:48.554191 (+ 245us) server_negotiation.cc:407] Connection header received
0924 23:14:48.554251 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:48.554252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:48.554276 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:48.554332 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:48.554463 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:48.555249 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:48.556299 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:48.556631 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:48.556682 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:48.557448 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:48.557450 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:48.557451 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:48.557456 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:48.557471 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:48.557476 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:48.557521 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:48.557721 (+ 200us) server_negotiation.cc:299] Negotiation successful
0924 23:14:48.557736 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 23:14:48.566301 1043 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:14:48.567049 1043 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:14:48.569533 1043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd480c771cef4034a0ecadf7d48acf4d: Generated new TSK 0
I20260924 23:14:48.569720 1043 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:14:48.575181 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 963
W20260924 23:14:48.578954 948 connection.cc:535] client connection to 127.0.0.1:32971 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32971): Connection reset by peer (error 104)
2026-09-24T23:14:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1242 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:14:48 dist-test-slave-m0z6 krb5kdc[1062](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:14:48 dist-test-slave-m0z6 krb5kdc[1062](info): set up 2 sockets
Sep 24 23:14:48 dist-test-slave-m0z6 krb5kdc[1062](info): commencing operation
krb5kdc: starting...
W20260924 23:14:49.522503 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:14:49 dist-test-slave-m0z6 krb5kdc[1062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:14:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:14:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 23:14:49.641274 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45651
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:49.722754 1075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:49.722988 1075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:49.723022 1075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:49.724229 1075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:14:49.724278 1075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:49.724298 1075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:49.724314 1075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:14:49.724328 1075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:14:49.725878 1075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45651
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:49.726143 1075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:49.726358 1075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:49.731072 1081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:49.731091 1080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:49.731067 1083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:49.731540 1075 server_base.cc:1034] running on GCE node
I20260924 23:14:49.731786 1075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:49.732079 1075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:49.733228 1075 hybrid_clock.cc:648] HybridClock initialized: now 1790291689733209 us; error 34 us; skew 500 ppm
Sep 24 23:14:49 dist-test-slave-m0z6 krb5kdc[1062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291689, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:49.735792 1075 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 23:14:49.736253 1075 webserver.cc:460] Webserver started at http://127.0.0.1:35249/ using document root <none> and password file <none>
I20260924 23:14:49.736459 1075 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:49.736506 1075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:49.736639 1075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:49.737701 1075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "ebe2d97c38194e26abe1428bac7a9802"
format_stamp: "Formatted at 2026-09-24 23:14:49 on dist-test-slave-m0z6"
server_key: "f8ca3a562df8f5aa03ebe835c3506f3e"
server_key_iv: "0bd3b21853f04e685686200cc8807fe9"
server_key_version: "encryptionkey@0"
I20260924 23:14:49.738036 1075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "ebe2d97c38194e26abe1428bac7a9802"
format_stamp: "Formatted at 2026-09-24 23:14:49 on dist-test-slave-m0z6"
server_key: "f8ca3a562df8f5aa03ebe835c3506f3e"
server_key_iv: "0bd3b21853f04e685686200cc8807fe9"
server_key_version: "encryptionkey@0"
I20260924 23:14:49.739413 1075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:14:49.740276 1090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:49.740429 1075 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:14:49.740509 1075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "ebe2d97c38194e26abe1428bac7a9802"
format_stamp: "Formatted at 2026-09-24 23:14:49 on dist-test-slave-m0z6"
server_key: "f8ca3a562df8f5aa03ebe835c3506f3e"
server_key_iv: "0bd3b21853f04e685686200cc8807fe9"
server_key_version: "encryptionkey@0"
I20260924 23:14:49.740600 1075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:49.743929 1075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:49.745972 1075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:49.746094 1075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:49.749948 1075 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45651
I20260924 23:14:49.749979 1142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45651 every 8 connection(s)
I20260924 23:14:49.750290 1075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:14:49.751164 1143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:49.753399 1143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802: Bootstrap starting.
I20260924 23:14:49.753952 1143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:49.754220 1143 log.cc:824] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:49.754914 1143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802: No bootstrap required, opened a new log
I20260924 23:14:49.756038 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 1075
I20260924 23:14:49.756155 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:14:49.756338 13136 external_mini_cluster.cc:1428] Setting key d2e0107c07d2df8029c1c21fe97a4514
I20260924 23:14:49.756820 1143 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe2d97c38194e26abe1428bac7a9802" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45651 } }
I20260924 23:14:49.757011 1143 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:49.757055 1143 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebe2d97c38194e26abe1428bac7a9802, State: Initialized, Role: FOLLOWER
I20260924 23:14:49.757192 1143 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe2d97c38194e26abe1428bac7a9802" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45651 } }
I20260924 23:14:49.757267 1143 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:14:49.757297 1143 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:14:49.757344 1143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:49.757860 1143 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe2d97c38194e26abe1428bac7a9802" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45651 } }
I20260924 23:14:49.757953 1143 leader_election.cc:302] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebe2d97c38194e26abe1428bac7a9802; no voters:
I20260924 23:14:49.758117 1143 leader_election.cc:288] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:14:49.758172 1148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:49.758286 1148 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802 [term 1 LEADER]: Becoming Leader. State: Replica: ebe2d97c38194e26abe1428bac7a9802, State: Running, Role: LEADER
I20260924 23:14:49.758363 1143 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:14:49.758368 1148 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe2d97c38194e26abe1428bac7a9802" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45651 } }
I20260924 23:14:49.758937 1147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:49.756855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57105 (local address 127.0.0.1:45651)
0924 23:14:49.757061 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:49.757072 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:49.757111 (+ 39us) server_negotiation.cc:407] Connection header received
0924 23:14:49.757232 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:49.757238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:49.757262 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:49.757399 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:49.758459 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:49.758462 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:49.758463 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:49.758467 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:49.758508 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:49.758522 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:49.758576 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:49.758664 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 23:14:49.758731 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260924 23:14:49.759145 1149 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebe2d97c38194e26abe1428bac7a9802" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe2d97c38194e26abe1428bac7a9802" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45651 } } }
I20260924 23:14:49.759233 1149 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:49.759305 1150 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebe2d97c38194e26abe1428bac7a9802. Latest consensus state: current_term: 1 leader_uuid: "ebe2d97c38194e26abe1428bac7a9802" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe2d97c38194e26abe1428bac7a9802" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45651 } } }
I20260924 23:14:49.759555 1158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:14:49.759582 1150 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:49.759845 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:14:49.760092 1158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:14:49.761704 1158 catalog_manager.cc:1347] Generated new cluster ID: 98fc3cf2f1fa40ecaadbb07a8043d376
I20260924 23:14:49.761754 1158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:14:49.763389 1147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:49.761273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45339 (local address 127.0.0.1:45651)
0924 23:14:49.761328 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:49.761330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:49.761436 (+ 106us) server_negotiation.cc:407] Connection header received
0924 23:14:49.761479 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:49.761480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:49.761484 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:49.761547 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:49.763195 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:49.763196 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:49.763196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:49.763200 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:49.763215 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:49.763219 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:49.763263 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:49.763316 (+ 53us) server_negotiation.cc:299] Negotiation successful
0924 23:14:49.763330 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 23:14:49.767554 1158 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:14:49.767719 1158 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:14:49.776602 1158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebe2d97c38194e26abe1428bac7a9802: Generated new TSK 0
I20260924 23:14:49.776822 1158 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:14:49.783032 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 1075
2026-09-24T23:14:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1208 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:14:49 dist-test-slave-m0z6 krb5kdc[1174](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:14:49 dist-test-slave-m0z6 krb5kdc[1174](info): set up 2 sockets
Sep 24 23:14:49 dist-test-slave-m0z6 krb5kdc[1174](info): commencing operation
krb5kdc: starting...
W20260924 23:14:50.746542 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:14:50 dist-test-slave-m0z6 krb5kdc[1174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:14:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:14:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 23:14:50.861083 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45239
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:50.942346 1187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:50.942584 1187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:50.942617 1187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:50.943895 1187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:14:50.943954 1187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:50.943974 1187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:50.943990 1187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:14:50.944006 1187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:14:50.945704 1187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:50.945981 1187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:50.946203 1187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:14:50.951503 1187 server_base.cc:1034] running on GCE node
W20260924 23:14:50.951635 1195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:50.951519 1193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:50.951759 1192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:50.952035 1187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:50.952368 1187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:50.953562 1187 hybrid_clock.cc:648] HybridClock initialized: now 1790291690953544 us; error 37 us; skew 500 ppm
Sep 24 23:14:50 dist-test-slave-m0z6 krb5kdc[1174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291690, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:50.956296 1187 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 23:14:50.956794 1187 webserver.cc:460] Webserver started at http://127.0.0.1:45587/ using document root <none> and password file <none>
I20260924 23:14:50.957010 1187 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:50.957057 1187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:50.957170 1187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:50.958302 1187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "c8716f8adebe497eabd436ab98f7acb5"
format_stamp: "Formatted at 2026-09-24 23:14:50 on dist-test-slave-m0z6"
server_key: "d84758ae6eb838f39b91e93d512a59e8"
server_key_iv: "b6dcc7daa562163be3986a18c232ef8f"
server_key_version: "encryptionkey@0"
I20260924 23:14:50.958642 1187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "c8716f8adebe497eabd436ab98f7acb5"
format_stamp: "Formatted at 2026-09-24 23:14:50 on dist-test-slave-m0z6"
server_key: "d84758ae6eb838f39b91e93d512a59e8"
server_key_iv: "b6dcc7daa562163be3986a18c232ef8f"
server_key_version: "encryptionkey@0"
I20260924 23:14:50.960119 1187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:50.960940 1202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:50.961148 1187 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:50.961246 1187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "c8716f8adebe497eabd436ab98f7acb5"
format_stamp: "Formatted at 2026-09-24 23:14:50 on dist-test-slave-m0z6"
server_key: "d84758ae6eb838f39b91e93d512a59e8"
server_key_iv: "b6dcc7daa562163be3986a18c232ef8f"
server_key_version: "encryptionkey@0"
I20260924 23:14:50.961319 1187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:50.979871 1187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:50.982189 1187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:50.982344 1187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:50.986280 1187 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45239
I20260924 23:14:50.986321 1254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45239 every 8 connection(s)
I20260924 23:14:50.986668 1187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:14:50.987566 1255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:50.989943 1255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5: Bootstrap starting.
I20260924 23:14:50.990583 1255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:50.990878 1255 log.cc:824] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:50.991537 1255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5: No bootstrap required, opened a new log
I20260924 23:14:50.993139 1255 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8716f8adebe497eabd436ab98f7acb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45239 } }
I20260924 23:14:50.993288 1255 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:50.993310 1255 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8716f8adebe497eabd436ab98f7acb5, State: Initialized, Role: FOLLOWER
I20260924 23:14:50.993448 1255 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8716f8adebe497eabd436ab98f7acb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45239 } }
I20260924 23:14:50.993511 1255 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:14:50.993532 1255 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:14:50.993564 1255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:50.994077 1255 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8716f8adebe497eabd436ab98f7acb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45239 } }
I20260924 23:14:50.994171 1255 leader_election.cc:302] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8716f8adebe497eabd436ab98f7acb5; no voters:
I20260924 23:14:50.994335 1255 leader_election.cc:288] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:14:50.994376 1258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:50.994542 1258 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5 [term 1 LEADER]: Becoming Leader. State: Replica: c8716f8adebe497eabd436ab98f7acb5, State: Running, Role: LEADER
I20260924 23:14:50.994577 1255 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:14:50.994632 1258 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8716f8adebe497eabd436ab98f7acb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45239 } }
I20260924 23:14:50.995009 1260 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8716f8adebe497eabd436ab98f7acb5. Latest consensus state: current_term: 1 leader_uuid: "c8716f8adebe497eabd436ab98f7acb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8716f8adebe497eabd436ab98f7acb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45239 } } }
I20260924 23:14:50.995095 1260 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:50.995139 1259 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8716f8adebe497eabd436ab98f7acb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8716f8adebe497eabd436ab98f7acb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45239 } } }
I20260924 23:14:50.995187 1259 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:50.995349 1266 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:14:50.995905 1266 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:14:50.996259 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 1187
I20260924 23:14:50.996363 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:14:50.996538 13136 external_mini_cluster.cc:1428] Setting key f26d7284449212d9b1bbc3177b0073c2
I20260924 23:14:50.997624 1266 catalog_manager.cc:1347] Generated new cluster ID: ccde4a818e164e0e81bdbf2da2c00572
I20260924 23:14:50.997673 1266 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 23:14:51 dist-test-slave-m0z6 krb5kdc[1174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 23:14:51.006713 1266 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:14:51.007395 1266 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:14:51.009056 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:50.997133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59585 (local address 127.0.0.1:45239)
0924 23:14:50.997297 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:50.997307 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:50.997341 (+ 34us) server_negotiation.cc:407] Connection header received
0924 23:14:50.997460 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:50.997468 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:50.997525 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:50.997795 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:50.998476 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:50.999895 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:51.000981 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:51.001614 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:51.001714 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:51.004732 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:51.004737 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:51.004737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:51.004752 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:51.006914 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:51.007344 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:51.007347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:51.007347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:51.007474 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:51.007699 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:51.007700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:51.007700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:51.008014 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:51.008153 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:51.008623 (+ 470us) server_negotiation.cc:299] Negotiation successful
0924 23:14:51.008788 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260924 23:14:51.009970 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:14:51.014855 1266 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8716f8adebe497eabd436ab98f7acb5: Generated new TSK 0
I20260924 23:14:51.015058 1266 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:14:51.018605 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:51.011584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43645 (local address 127.0.0.1:45239)
0924 23:14:51.011620 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:51.011622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:51.011629 (+ 7us) server_negotiation.cc:407] Connection header received
0924 23:14:51.011652 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:51.011652 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:51.011662 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:51.011713 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:51.012320 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:51.013116 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:51.014164 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:51.014546 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:51.014615 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:51.016174 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:51.016176 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:51.016176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:51.016188 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:51.017556 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:51.017931 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:51.017932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:51.017932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:51.017979 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:51.018182 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:51.018182 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:51.018183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:51.018281 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:51.018370 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:51.018478 (+ 108us) server_negotiation.cc:299] Negotiation successful
0924 23:14:51.018542 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 23:14:51.020658 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 1187
2026-09-24T23:14:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1237 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:14:51 dist-test-slave-m0z6 krb5kdc[1286](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:14:51 dist-test-slave-m0z6 krb5kdc[1286](info): set up 2 sockets
Sep 24 23:14:51 dist-test-slave-m0z6 krb5kdc[1286](info): commencing operation
krb5kdc: starting...
W20260924 23:14:51.978433 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:14:52 dist-test-slave-m0z6 krb5kdc[1286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:14:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:14:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 23:14:52.097783 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45819
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45819
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:52.178381 1299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:52.178601 1299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:52.178627 1299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:52.179872 1299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:14:52.179924 1299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:52.179937 1299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:52.179948 1299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:14:52.179967 1299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:14:52.181591 1299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:52.181838 1299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:52.182101 1299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:52.187047 1304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:52.187054 1307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:52.187122 1305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:52.187963 1299 server_base.cc:1034] running on GCE node
I20260924 23:14:52.188351 1299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:52.188692 1299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:52.189884 1299 hybrid_clock.cc:648] HybridClock initialized: now 1790291692189862 us; error 42 us; skew 500 ppm
Sep 24 23:14:52 dist-test-slave-m0z6 krb5kdc[1286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291692, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:52.192672 1299 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 23:14:52.193181 1299 webserver.cc:460] Webserver started at http://127.0.0.1:39811/ using document root <none> and password file <none>
I20260924 23:14:52.193411 1299 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:52.193459 1299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:52.193573 1299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:52.194681 1299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "5714ddb0da68432ab4bf7f8d8bf9f671"
format_stamp: "Formatted at 2026-09-24 23:14:52 on dist-test-slave-m0z6"
server_key: "542849368ec7d0b9927307c85c8b5806"
server_key_iv: "43c09fc36c63dd40e0c90e7d53da2064"
server_key_version: "encryptionkey@0"
I20260924 23:14:52.195089 1299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "5714ddb0da68432ab4bf7f8d8bf9f671"
format_stamp: "Formatted at 2026-09-24 23:14:52 on dist-test-slave-m0z6"
server_key: "542849368ec7d0b9927307c85c8b5806"
server_key_iv: "43c09fc36c63dd40e0c90e7d53da2064"
server_key_version: "encryptionkey@0"
I20260924 23:14:52.196414 1299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:14:52.197182 1314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:52.197345 1299 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 23:14:52.197431 1299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "5714ddb0da68432ab4bf7f8d8bf9f671"
format_stamp: "Formatted at 2026-09-24 23:14:52 on dist-test-slave-m0z6"
server_key: "542849368ec7d0b9927307c85c8b5806"
server_key_iv: "43c09fc36c63dd40e0c90e7d53da2064"
server_key_version: "encryptionkey@0"
I20260924 23:14:52.197497 1299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:52.218339 1299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:52.220685 1299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:52.220831 1299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:52.224733 1299 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45819
I20260924 23:14:52.224769 1366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45819 every 8 connection(s)
I20260924 23:14:52.225113 1299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:14:52.225973 1367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:52.228368 1367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671: Bootstrap starting.
I20260924 23:14:52.228921 1367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:52.229183 1367 log.cc:824] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:52.229910 1367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671: No bootstrap required, opened a new log
I20260924 23:14:52.231559 1367 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5714ddb0da68432ab4bf7f8d8bf9f671" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45819 } }
I20260924 23:14:52.231730 1367 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:52.231758 1367 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5714ddb0da68432ab4bf7f8d8bf9f671, State: Initialized, Role: FOLLOWER
I20260924 23:14:52.231880 1367 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5714ddb0da68432ab4bf7f8d8bf9f671" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45819 } }
I20260924 23:14:52.231933 1367 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:14:52.231972 1367 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:14:52.232019 1367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:52.232605 1367 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5714ddb0da68432ab4bf7f8d8bf9f671" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45819 } }
I20260924 23:14:52.232704 1367 leader_election.cc:302] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5714ddb0da68432ab4bf7f8d8bf9f671; no voters:
I20260924 23:14:52.232882 1367 leader_election.cc:288] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:14:52.232911 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 1299
I20260924 23:14:52.232923 1370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:52.233032 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:14:52.233172 1367 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:14:52.233230 13136 external_mini_cluster.cc:1428] Setting key 7e02631ca4edfa93b8592de276a1722c
I20260924 23:14:52.233204 1370 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671 [term 1 LEADER]: Becoming Leader. State: Replica: 5714ddb0da68432ab4bf7f8d8bf9f671, State: Running, Role: LEADER
I20260924 23:14:52.233476 1370 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5714ddb0da68432ab4bf7f8d8bf9f671" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45819 } }
I20260924 23:14:52.234035 1372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5714ddb0da68432ab4bf7f8d8bf9f671. Latest consensus state: current_term: 1 leader_uuid: "5714ddb0da68432ab4bf7f8d8bf9f671" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5714ddb0da68432ab4bf7f8d8bf9f671" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45819 } } }
I20260924 23:14:52.234174 1372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:52.234292 1371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5714ddb0da68432ab4bf7f8d8bf9f671" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5714ddb0da68432ab4bf7f8d8bf9f671" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45819 } } }
I20260924 23:14:52.234366 1371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:52.234719 1381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:14:52.235333 1381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:14:52.236768 1381 catalog_manager.cc:1347] Generated new cluster ID: 2523780e0e514235aa64f83f1927a613
I20260924 23:14:52.236817 1381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:14:52.240191 1380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:52.233910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51991 (local address 127.0.0.1:45819)
0924 23:14:52.234146 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:52.234156 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:52.234505 (+ 349us) server_negotiation.cc:407] Connection header received
0924 23:14:52.234788 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:52.234797 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:52.234860 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:52.235013 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:52.235328 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:52.236741 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:52.237880 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:52.238570 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:52.238731 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:52.239435 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:52.239439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:52.239440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:52.239669 (+ 229us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:52.239715 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:52.239732 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:52.239803 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:52.239871 (+ 68us) server_negotiation.cc:299] Negotiation successful
0924 23:14:52.239961 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":154,"threads_started":1}
I20260924 23:14:52.241341 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:14:52.246390 1380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:52.242908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53533 (local address 127.0.0.1:45819)
0924 23:14:52.242961 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:52.242963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:52.243113 (+ 150us) server_negotiation.cc:407] Connection header received
0924 23:14:52.243166 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:52.243167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:52.243177 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:52.243246 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:52.243371 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:52.244133 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:52.245102 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:52.245421 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:52.245487 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:52.246147 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:52.246149 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:52.246149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:52.246154 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:52.246170 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:52.246174 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:52.246229 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:52.246273 (+ 44us) server_negotiation.cc:299] Negotiation successful
0924 23:14:52.246302 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 23:14:52.252370 1381 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:14:52.253165 1381 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:14:52.258028 1381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5714ddb0da68432ab4bf7f8d8bf9f671: Generated new TSK 0
I20260924 23:14:52.258252 1381 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:14:52.266041 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 1299
2026-09-24T23:14:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1244 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:14:52 dist-test-slave-m0z6 krb5kdc[1398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:14:52 dist-test-slave-m0z6 krb5kdc[1398](info): set up 2 sockets
Sep 24 23:14:52 dist-test-slave-m0z6 krb5kdc[1398](info): commencing operation
krb5kdc: starting...
W20260924 23:14:53.218580 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:14:53 dist-test-slave-m0z6 krb5kdc[1398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:14:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:14:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 23:14:53.335693 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32907
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:53.415819 1411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:53.416050 1411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:53.416080 1411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:53.417277 1411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:14:53.417328 1411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:53.417347 1411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:53.417363 1411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:14:53.417378 1411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:14:53.418910 1411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:53.419163 1411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:53.419371 1411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:53.424031 1416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:53.424026 1419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:53.424026 1417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:53.424360 1411 server_base.cc:1034] running on GCE node
I20260924 23:14:53.424640 1411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:53.424942 1411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:53.426100 1411 hybrid_clock.cc:648] HybridClock initialized: now 1790291693426078 us; error 39 us; skew 500 ppm
Sep 24 23:14:53 dist-test-slave-m0z6 krb5kdc[1398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291693, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:53.428647 1411 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 23:14:53.429102 1411 webserver.cc:460] Webserver started at http://127.0.0.1:45903/ using document root <none> and password file <none>
I20260924 23:14:53.429296 1411 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:53.429335 1411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:53.429430 1411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:53.430450 1411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "9dcedb51206f40579c6d03b3e9eeaef9"
format_stamp: "Formatted at 2026-09-24 23:14:53 on dist-test-slave-m0z6"
server_key: "45bf14d49f5e2e6d5c7ac1af5ea3e3e9"
server_key_iv: "65358e9851aa9e75b15cf17631816967"
server_key_version: "encryptionkey@0"
I20260924 23:14:53.430835 1411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "9dcedb51206f40579c6d03b3e9eeaef9"
format_stamp: "Formatted at 2026-09-24 23:14:53 on dist-test-slave-m0z6"
server_key: "45bf14d49f5e2e6d5c7ac1af5ea3e3e9"
server_key_iv: "65358e9851aa9e75b15cf17631816967"
server_key_version: "encryptionkey@0"
I20260924 23:14:53.432127 1411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:53.432857 1426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:53.433048 1411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 23:14:53.433132 1411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "9dcedb51206f40579c6d03b3e9eeaef9"
format_stamp: "Formatted at 2026-09-24 23:14:53 on dist-test-slave-m0z6"
server_key: "45bf14d49f5e2e6d5c7ac1af5ea3e3e9"
server_key_iv: "65358e9851aa9e75b15cf17631816967"
server_key_version: "encryptionkey@0"
I20260924 23:14:53.433198 1411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:53.436575 1411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:53.438726 1411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:53.438865 1411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:53.442488 1411 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32907
I20260924 23:14:53.442519 1478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32907 every 8 connection(s)
I20260924 23:14:53.442876 1411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:14:53.443730 1479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:53.446017 1479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9: Bootstrap starting.
I20260924 23:14:53.446547 1479 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:53.446834 1479 log.cc:824] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:53.447446 1479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9: No bootstrap required, opened a new log
I20260924 23:14:53.449257 1479 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dcedb51206f40579c6d03b3e9eeaef9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32907 } }
I20260924 23:14:53.449401 1479 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:53.449433 1479 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dcedb51206f40579c6d03b3e9eeaef9, State: Initialized, Role: FOLLOWER
I20260924 23:14:53.449549 1479 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dcedb51206f40579c6d03b3e9eeaef9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32907 } }
I20260924 23:14:53.449607 1479 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:14:53.449632 1479 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:14:53.449663 1479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:53.450122 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 1411
I20260924 23:14:53.450235 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:14:53.450261 1479 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dcedb51206f40579c6d03b3e9eeaef9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32907 } }
I20260924 23:14:53.450366 1479 leader_election.cc:302] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9dcedb51206f40579c6d03b3e9eeaef9; no voters:
I20260924 23:14:53.450435 13136 external_mini_cluster.cc:1428] Setting key 6f953efeb57404477650eb857489c9c3
I20260924 23:14:53.450539 1479 leader_election.cc:288] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:14:53.450569 1482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:53.450810 1482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9 [term 1 LEADER]: Becoming Leader. State: Replica: 9dcedb51206f40579c6d03b3e9eeaef9, State: Running, Role: LEADER
I20260924 23:14:53.450959 1482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dcedb51206f40579c6d03b3e9eeaef9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32907 } }
I20260924 23:14:53.450820 1479 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:14:53.451345 1484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9dcedb51206f40579c6d03b3e9eeaef9. Latest consensus state: current_term: 1 leader_uuid: "9dcedb51206f40579c6d03b3e9eeaef9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dcedb51206f40579c6d03b3e9eeaef9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32907 } } }
I20260924 23:14:53.451422 1484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:53.451468 1483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9dcedb51206f40579c6d03b3e9eeaef9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dcedb51206f40579c6d03b3e9eeaef9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32907 } } }
I20260924 23:14:53.451514 1483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:53.451701 1488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:14:53.452229 1488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:14:53.452942 1486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:53.450961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36971 (local address 127.0.0.1:32907)
0924 23:14:53.451177 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:53.451187 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:53.451221 (+ 34us) server_negotiation.cc:407] Connection header received
0924 23:14:53.451345 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:53.451352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:53.451384 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:53.451831 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:53.452301 (+ 470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:53.452305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:53.452305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:53.452308 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:53.452343 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:53.452359 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:53.452692 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:53.452727 (+ 35us) server_negotiation.cc:299] Negotiation successful
0924 23:14:53.452770 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":71,"threads_started":1}
I20260924 23:14:53.453831 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:14:53.453902 1488 catalog_manager.cc:1347] Generated new cluster ID: c75377b7ff4646d7b5afcf6ad81f03dc
I20260924 23:14:53.453941 1488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:14:53.456318 1486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:53.455304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51613 (local address 127.0.0.1:32907)
0924 23:14:53.455341 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:53.455343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:53.455475 (+ 132us) server_negotiation.cc:407] Connection header received
0924 23:14:53.455536 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:53.455537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:53.455543 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:53.455589 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:53.456136 (+ 547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:53.456137 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:53.456137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:53.456140 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:53.456154 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:53.456157 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:53.456203 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:53.456248 (+ 45us) server_negotiation.cc:299] Negotiation successful
0924 23:14:53.456260 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 23:14:53.466998 1488 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:14:53.467195 1488 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:14:53.473330 1488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9dcedb51206f40579c6d03b3e9eeaef9: Generated new TSK 0
I20260924 23:14:53.473556 1488 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:14:53.477241 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 1411
2026-09-24T23:14:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1211 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:14:53 dist-test-slave-m0z6 krb5kdc[1510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:14:53 dist-test-slave-m0z6 krb5kdc[1510](info): set up 2 sockets
Sep 24 23:14:53 dist-test-slave-m0z6 krb5kdc[1510](info): commencing operation
krb5kdc: starting...
W20260924 23:14:54.442612 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:14:54 dist-test-slave-m0z6 krb5kdc[1510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:14:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:14:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 23:14:54.559576 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46039
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:54.639868 1523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:54.640106 1523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:54.640138 1523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:54.641327 1523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:14:54.641376 1523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:54.641394 1523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:54.641410 1523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:14:54.641426 1523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:14:54.643007 1523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:54.643253 1523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:54.643472 1523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:54.648250 1528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:54.648295 1529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:54.648285 1531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:54.648492 1523 server_base.cc:1034] running on GCE node
I20260924 23:14:54.648742 1523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:54.649039 1523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:54.650190 1523 hybrid_clock.cc:648] HybridClock initialized: now 1790291694650179 us; error 35 us; skew 500 ppm
Sep 24 23:14:54 dist-test-slave-m0z6 krb5kdc[1510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291694, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:54.652673 1523 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 23:14:54.653106 1523 webserver.cc:460] Webserver started at http://127.0.0.1:38877/ using document root <none> and password file <none>
I20260924 23:14:54.653307 1523 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:54.653357 1523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:54.653481 1523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:54.654534 1523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "4bab9592c19c40349064eb49940a8844"
format_stamp: "Formatted at 2026-09-24 23:14:54 on dist-test-slave-m0z6"
server_key: "b87eec9e5af50c8a5f8f1aca68bd7235"
server_key_iv: "d661d8e5f87e3a04f7301b7cadc808d1"
server_key_version: "encryptionkey@0"
I20260924 23:14:54.654906 1523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "4bab9592c19c40349064eb49940a8844"
format_stamp: "Formatted at 2026-09-24 23:14:54 on dist-test-slave-m0z6"
server_key: "b87eec9e5af50c8a5f8f1aca68bd7235"
server_key_iv: "d661d8e5f87e3a04f7301b7cadc808d1"
server_key_version: "encryptionkey@0"
I20260924 23:14:54.656185 1523 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:14:54.656886 1538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:54.657045 1523 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:54.657142 1523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "4bab9592c19c40349064eb49940a8844"
format_stamp: "Formatted at 2026-09-24 23:14:54 on dist-test-slave-m0z6"
server_key: "b87eec9e5af50c8a5f8f1aca68bd7235"
server_key_iv: "d661d8e5f87e3a04f7301b7cadc808d1"
server_key_version: "encryptionkey@0"
I20260924 23:14:54.657212 1523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:54.688638 1523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:54.691134 1523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:54.691293 1523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:54.695374 1523 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46039
I20260924 23:14:54.695428 1590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46039 every 8 connection(s)
I20260924 23:14:54.695758 1523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:14:54.696635 1591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:54.699083 1591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844: Bootstrap starting.
I20260924 23:14:54.699661 1591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:54.699915 1591 log.cc:824] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:54.700572 1591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844: No bootstrap required, opened a new log
I20260924 23:14:54.702157 1591 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bab9592c19c40349064eb49940a8844" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46039 } }
I20260924 23:14:54.702299 1591 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:54.702322 1591 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bab9592c19c40349064eb49940a8844, State: Initialized, Role: FOLLOWER
I20260924 23:14:54.702455 1591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bab9592c19c40349064eb49940a8844" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46039 } }
I20260924 23:14:54.702518 1591 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:14:54.702539 1591 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:14:54.702567 1591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:54.703127 1591 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bab9592c19c40349064eb49940a8844" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46039 } }
I20260924 23:14:54.703219 1591 leader_election.cc:302] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bab9592c19c40349064eb49940a8844; no voters:
I20260924 23:14:54.703387 1591 leader_election.cc:288] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:14:54.703419 1594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:54.703581 1594 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844 [term 1 LEADER]: Becoming Leader. State: Replica: 4bab9592c19c40349064eb49940a8844, State: Running, Role: LEADER
I20260924 23:14:54.703645 1591 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:14:54.703699 1594 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bab9592c19c40349064eb49940a8844" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46039 } }
I20260924 23:14:54.704062 1596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bab9592c19c40349064eb49940a8844. Latest consensus state: current_term: 1 leader_uuid: "4bab9592c19c40349064eb49940a8844" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bab9592c19c40349064eb49940a8844" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46039 } } }
I20260924 23:14:54.704190 1595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4bab9592c19c40349064eb49940a8844" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bab9592c19c40349064eb49940a8844" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46039 } } }
I20260924 23:14:54.704196 1596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:54.704303 1595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:54.704578 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 1523
I20260924 23:14:54.704715 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:14:54.704797 1605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:14:54.704906 13136 external_mini_cluster.cc:1428] Setting key 9254c6b470df26a075a530e04297581f
I20260924 23:14:54.705390 1605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:14:54.707116 1605 catalog_manager.cc:1347] Generated new cluster ID: 113b0cecdedb4b849686bf7d7220b247
I20260924 23:14:54.707172 1605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 23:14:54 dist-test-slave-m0z6 krb5kdc[1510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 23:14:54.715027 1613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:54.705383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:41507 (local address 127.0.0.1:46039)
0924 23:14:54.705576 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:54.705586 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:54.705625 (+ 39us) server_negotiation.cc:407] Connection header received
0924 23:14:54.705724 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:54.705730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:54.705776 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:54.705894 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:54.706793 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:54.707794 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:54.708520 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:54.708947 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:54.709042 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:54.711384 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:54.711389 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:54.711389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:54.711402 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:54.713028 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:54.713528 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:54.713532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:54.713532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:54.713635 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:54.713841 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:54.713842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:54.713842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:54.714242 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:54.714414 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:54.714597 (+ 183us) server_negotiation.cc:299] Negotiation successful
0924 23:14:54.714840 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":78,"threads_started":1}
I20260924 23:14:54.715922 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:14:54.716478 1605 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:14:54.717155 1605 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:14:54.721992 1605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bab9592c19c40349064eb49940a8844: Generated new TSK 0
I20260924 23:14:54.722205 1605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:14:54.723256 1613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:54.717409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:37223 (local address 127.0.0.1:46039)
0924 23:14:54.717443 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:54.717445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:54.717454 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:14:54.717482 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:54.717483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:54.717492 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:54.717538 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:54.718034 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:54.718569 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:54.719332 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:54.719573 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:54.719629 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:54.720765 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:54.720766 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:54.720766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:54.720773 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:54.722147 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:54.722498 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:54.722499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:54.722500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:54.722567 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:54.722784 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:54.722785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:54.722785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:54.722892 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:54.722990 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:54.723108 (+ 118us) server_negotiation.cc:299] Negotiation successful
0924 23:14:54.723167 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":13,"server-negotiator.queue_time_us":18}
I20260924 23:14:54.725575 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 1523
W20260924 23:14:54.729118 1508 connection.cc:535] client connection to 127.0.0.1:46039 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46039): Connection reset by peer (error 104)
2026-09-24T23:14:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1248 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:14:54 dist-test-slave-m0z6 krb5kdc[1622](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:14:54 dist-test-slave-m0z6 krb5kdc[1622](info): set up 2 sockets
Sep 24 23:14:54 dist-test-slave-m0z6 krb5kdc[1622](info): commencing operation
krb5kdc: starting...
W20260924 23:14:55.674847 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:14:55 dist-test-slave-m0z6 krb5kdc[1622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:14:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:14:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 23:14:55.790057 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45573
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:55.870283 1635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:55.870523 1635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:55.870546 1635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:55.871845 1635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:14:55.871899 1635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:55.871912 1635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:55.871924 1635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:14:55.871935 1635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:14:55.873554 1635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:55.873829 1635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:55.874063 1635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:55.879021 1643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:55.879015 1641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:55.879258 1635 server_base.cc:1034] running on GCE node
W20260924 23:14:55.879021 1640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:55.879565 1635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:55.879879 1635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:55.881119 1635 hybrid_clock.cc:648] HybridClock initialized: now 1790291695881099 us; error 37 us; skew 500 ppm
Sep 24 23:14:55 dist-test-slave-m0z6 krb5kdc[1622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291695, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:55.883862 1635 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 23:14:55.884419 1635 webserver.cc:460] Webserver started at http://127.0.0.1:38197/ using document root <none> and password file <none>
I20260924 23:14:55.884636 1635 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:55.884686 1635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:55.884814 1635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:55.885913 1635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "3a0ce03ed52f4adbb78a53b00e9a559d"
format_stamp: "Formatted at 2026-09-24 23:14:55 on dist-test-slave-m0z6"
server_key: "2fc8d5a8a22adc8c38861cd24b68146e"
server_key_iv: "19067a336e2409c1e974dd26268e2403"
server_key_version: "encryptionkey@0"
I20260924 23:14:55.886291 1635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "3a0ce03ed52f4adbb78a53b00e9a559d"
format_stamp: "Formatted at 2026-09-24 23:14:55 on dist-test-slave-m0z6"
server_key: "2fc8d5a8a22adc8c38861cd24b68146e"
server_key_iv: "19067a336e2409c1e974dd26268e2403"
server_key_version: "encryptionkey@0"
I20260924 23:14:55.887810 1635 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:14:55.888767 1650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:55.889056 1635 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:14:55.889156 1635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "3a0ce03ed52f4adbb78a53b00e9a559d"
format_stamp: "Formatted at 2026-09-24 23:14:55 on dist-test-slave-m0z6"
server_key: "2fc8d5a8a22adc8c38861cd24b68146e"
server_key_iv: "19067a336e2409c1e974dd26268e2403"
server_key_version: "encryptionkey@0"
I20260924 23:14:55.889232 1635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:55.902541 1635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:55.904938 1635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:55.905098 1635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:55.909375 1635 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45573
I20260924 23:14:55.909432 1702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45573 every 8 connection(s)
I20260924 23:14:55.909775 1635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:14:55.910655 1703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:55.913230 1703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d: Bootstrap starting.
I20260924 23:14:55.913849 1703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:55.914136 1703 log.cc:824] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:55.914924 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 1635
I20260924 23:14:55.915058 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:14:55.915195 1703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d: No bootstrap required, opened a new log
I20260924 23:14:55.915248 13136 external_mini_cluster.cc:1428] Setting key 05e2ff828800f6a612ac36f861423e44
I20260924 23:14:55.917780 1703 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a0ce03ed52f4adbb78a53b00e9a559d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45573 } }
I20260924 23:14:55.917951 1703 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:55.917975 1703 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a0ce03ed52f4adbb78a53b00e9a559d, State: Initialized, Role: FOLLOWER
I20260924 23:14:55.918111 1703 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a0ce03ed52f4adbb78a53b00e9a559d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45573 } }
I20260924 23:14:55.918179 1703 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:14:55.918200 1703 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:14:55.918246 1703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:55.918892 1703 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a0ce03ed52f4adbb78a53b00e9a559d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45573 } }
I20260924 23:14:55.918996 1703 leader_election.cc:302] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a0ce03ed52f4adbb78a53b00e9a559d; no voters:
I20260924 23:14:55.919159 1703 leader_election.cc:288] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:14:55.919190 1708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:55.919338 1708 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d [term 1 LEADER]: Becoming Leader. State: Replica: 3a0ce03ed52f4adbb78a53b00e9a559d, State: Running, Role: LEADER
I20260924 23:14:55.919448 1703 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:14:55.919492 1708 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a0ce03ed52f4adbb78a53b00e9a559d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45573 } }
I20260924 23:14:55.920118 1709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a0ce03ed52f4adbb78a53b00e9a559d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a0ce03ed52f4adbb78a53b00e9a559d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45573 } } }
I20260924 23:14:55.920210 1709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:55.920645 1710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a0ce03ed52f4adbb78a53b00e9a559d. Latest consensus state: current_term: 1 leader_uuid: "3a0ce03ed52f4adbb78a53b00e9a559d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a0ce03ed52f4adbb78a53b00e9a559d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45573 } } }
I20260924 23:14:55.920902 1710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:55.921118 1707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:55.915818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:43757 (local address 127.0.0.1:45573)
0924 23:14:55.916038 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:55.916047 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:55.916075 (+ 28us) server_negotiation.cc:407] Connection header received
0924 23:14:55.916525 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:55.916533 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:55.916589 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:55.916718 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:55.916932 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:55.917809 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:55.918731 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:55.919208 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:55.919316 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:55.920244 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:55.920249 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:55.920249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:55.920465 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:55.920521 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:55.920537 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:55.920624 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:55.920886 (+ 262us) server_negotiation.cc:299] Negotiation successful
0924 23:14:55.920918 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":99,"threads_started":1}
W20260924 23:14:55.921423 1724 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:14:55.921530 1724 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:14:55.921787 1717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:14:55.922009 1717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:14:55.922163 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:14:55.923496 1717 catalog_manager.cc:1347] Generated new cluster ID: ec2973fe000541088e5de02f123186f8
I20260924 23:14:55.923561 1717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:14:55.927646 1707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:55.923624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:52489 (local address 127.0.0.1:45573)
0924 23:14:55.923666 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:55.923669 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:55.923908 (+ 239us) server_negotiation.cc:407] Connection header received
0924 23:14:55.923983 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:55.923984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:55.924002 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:55.924052 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:55.924201 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:55.924950 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:55.926120 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:55.926442 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:55.926525 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:55.927341 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:55.927342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:55.927343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:55.927349 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:55.927370 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:55.927376 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:55.927440 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:55.927508 (+ 68us) server_negotiation.cc:299] Negotiation successful
0924 23:14:55.927521 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 23:14:55.954775 1717 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:14:55.955365 1717 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:14:55.958122 1717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a0ce03ed52f4adbb78a53b00e9a559d: Generated new TSK 0
I20260924 23:14:55.958320 1717 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:14:55.969058 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 1635
W20260924 23:14:55.972995 1620 connection.cc:535] client connection to 127.0.0.1:45573 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45573): Connection reset by peer (error 104)
2026-09-24T23:14:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1243 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:14:56 dist-test-slave-m0z6 krb5kdc[1734](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:14:56 dist-test-slave-m0z6 krb5kdc[1734](info): set up 2 sockets
Sep 24 23:14:56 dist-test-slave-m0z6 krb5kdc[1734](info): commencing operation
krb5kdc: starting...
W20260924 23:14:56.918633 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:14:56 dist-test-slave-m0z6 krb5kdc[1734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:14:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:14:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 23:14:57.042510 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36399
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:57.124347 1747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:57.124580 1747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:57.124612 1747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:57.125828 1747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:14:57.125876 1747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:57.125895 1747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:57.125910 1747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:14:57.125933 1747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:14:57.127498 1747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:57.127744 1747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:57.127964 1747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:57.132728 1753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:57.132737 1752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:57.132767 1747 server_base.cc:1034] running on GCE node
W20260924 23:14:57.132776 1755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:57.133193 1747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:57.133471 1747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:57.134654 1747 hybrid_clock.cc:648] HybridClock initialized: now 1790291697134635 us; error 56 us; skew 500 ppm
Sep 24 23:14:57 dist-test-slave-m0z6 krb5kdc[1734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291697, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:57.137389 1747 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 23:14:57.137851 1747 webserver.cc:460] Webserver started at http://127.0.0.1:34307/ using document root <none> and password file <none>
I20260924 23:14:57.138064 1747 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:57.138116 1747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:57.138236 1747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:57.139375 1747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "f8b53fcaa400435da8f947705e4fbc6e"
format_stamp: "Formatted at 2026-09-24 23:14:57 on dist-test-slave-m0z6"
server_key: "3edf6ff9cacb07898d5543bce2eae577"
server_key_iv: "085e213c0da10926cef6fcb763562b51"
server_key_version: "encryptionkey@0"
I20260924 23:14:57.139714 1747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "f8b53fcaa400435da8f947705e4fbc6e"
format_stamp: "Formatted at 2026-09-24 23:14:57 on dist-test-slave-m0z6"
server_key: "3edf6ff9cacb07898d5543bce2eae577"
server_key_iv: "085e213c0da10926cef6fcb763562b51"
server_key_version: "encryptionkey@0"
I20260924 23:14:57.141058 1747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:14:57.141901 1762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:57.142115 1747 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:57.142199 1747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "f8b53fcaa400435da8f947705e4fbc6e"
format_stamp: "Formatted at 2026-09-24 23:14:57 on dist-test-slave-m0z6"
server_key: "3edf6ff9cacb07898d5543bce2eae577"
server_key_iv: "085e213c0da10926cef6fcb763562b51"
server_key_version: "encryptionkey@0"
I20260924 23:14:57.142271 1747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:57.145663 1747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:57.147923 1747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:57.148039 1747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:57.151873 1747 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36399
I20260924 23:14:57.151978 1814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36399 every 8 connection(s)
I20260924 23:14:57.152279 1747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:14:57.153198 1815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:57.155874 1815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e: Bootstrap starting.
I20260924 23:14:57.156487 1815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:57.156767 1815 log.cc:824] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:57.157328 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 1747
I20260924 23:14:57.157485 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:14:57.157672 13136 external_mini_cluster.cc:1428] Setting key 14f545d3e0e12da3a77f6996c8c0cf5d
I20260924 23:14:57.157670 1815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e: No bootstrap required, opened a new log
I20260924 23:14:57.160156 1815 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8b53fcaa400435da8f947705e4fbc6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36399 } }
I20260924 23:14:57.160334 1815 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:57.160358 1815 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8b53fcaa400435da8f947705e4fbc6e, State: Initialized, Role: FOLLOWER
I20260924 23:14:57.160481 1815 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8b53fcaa400435da8f947705e4fbc6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36399 } }
I20260924 23:14:57.160562 1815 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:14:57.160563 1819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:57.158248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:56571 (local address 127.0.0.1:36399)
0924 23:14:57.158409 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:57.158417 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:57.158443 (+ 26us) server_negotiation.cc:407] Connection header received
0924 23:14:57.159138 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:57.159145 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:57.159177 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:57.159301 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:57.159936 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:57.159939 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:57.159940 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:57.159943 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:57.159987 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:57.159998 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:57.160048 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:57.160129 (+ 81us) server_negotiation.cc:299] Negotiation successful
0924 23:14:57.160168 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260924 23:14:57.160585 1815 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:14:57.160691 1815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:57.161489 1815 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8b53fcaa400435da8f947705e4fbc6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36399 } }
I20260924 23:14:57.161583 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:14:57.161604 1815 leader_election.cc:302] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8b53fcaa400435da8f947705e4fbc6e; no voters:
I20260924 23:14:57.161769 1815 leader_election.cc:288] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:14:57.161810 1820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:57.161957 1820 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e [term 1 LEADER]: Becoming Leader. State: Replica: f8b53fcaa400435da8f947705e4fbc6e, State: Running, Role: LEADER
I20260924 23:14:57.162011 1815 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:14:57.162087 1820 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8b53fcaa400435da8f947705e4fbc6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36399 } }
I20260924 23:14:57.162410 1820 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8b53fcaa400435da8f947705e4fbc6e. Latest consensus state: current_term: 1 leader_uuid: "f8b53fcaa400435da8f947705e4fbc6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8b53fcaa400435da8f947705e4fbc6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36399 } } }
I20260924 23:14:57.162616 1820 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:57.162551 1821 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8b53fcaa400435da8f947705e4fbc6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8b53fcaa400435da8f947705e4fbc6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36399 } } }
I20260924 23:14:57.162873 1821 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:57.163478 1830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:14:57.164529 1819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:57.163305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:52791 (local address 127.0.0.1:36399)
0924 23:14:57.163336 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:57.163338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:57.163469 (+ 131us) server_negotiation.cc:407] Connection header received
0924 23:14:57.163604 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:57.163605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:57.163609 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:57.163637 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:57.164318 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:57.164319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:57.164319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:57.164324 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:57.164342 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:57.164347 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:57.164406 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:57.164459 (+ 53us) server_negotiation.cc:299] Negotiation successful
0924 23:14:57.164471 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 23:14:57.165328 1830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:14:57.166797 1830 catalog_manager.cc:1347] Generated new cluster ID: e8492b8161b34d458567480aa73af315
I20260924 23:14:57.166847 1830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:14:57.180851 1830 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:14:57.181046 1830 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:14:57.186532 1830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8b53fcaa400435da8f947705e4fbc6e: Generated new TSK 0
I20260924 23:14:57.186784 1830 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:14:57.192219 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 1747
2026-09-24T23:14:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1223 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:14:57 dist-test-slave-m0z6 krb5kdc[1846](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:14:57 dist-test-slave-m0z6 krb5kdc[1846](info): set up 2 sockets
Sep 24 23:14:57 dist-test-slave-m0z6 krb5kdc[1846](info): commencing operation
krb5kdc: starting...
W20260924 23:14:58.150617 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:14:58 dist-test-slave-m0z6 krb5kdc[1846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:14:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:14:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 23:14:58.270753 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43969
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43969
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:58.352645 1859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:58.352845 1859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:58.352869 1859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:58.354024 1859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:14:58.354063 1859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:58.354077 1859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:58.354089 1859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:14:58.354100 1859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:14:58.355571 1859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43969
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:58.355777 1859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:58.355954 1859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:58.360760 1864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:58.360760 1867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:58.360760 1865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:58.360872 1859 server_base.cc:1034] running on GCE node
I20260924 23:14:58.361236 1859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:58.361539 1859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:58.362733 1859 hybrid_clock.cc:648] HybridClock initialized: now 1790291698362713 us; error 36 us; skew 500 ppm
Sep 24 23:14:58 dist-test-slave-m0z6 krb5kdc[1846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291698, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:58.365195 1859 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 23:14:58.365653 1859 webserver.cc:460] Webserver started at http://127.0.0.1:43021/ using document root <none> and password file <none>
I20260924 23:14:58.365852 1859 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:58.365891 1859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:58.365983 1859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:58.367146 1859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "b918721d1d2c42edbdf7437d72970347"
format_stamp: "Formatted at 2026-09-24 23:14:58 on dist-test-slave-m0z6"
server_key: "de92ec7b15e3a0ad37f747f1449eb4ab"
server_key_iv: "03d90e920f6d09dbdef75a9a788b2a01"
server_key_version: "encryptionkey@0"
I20260924 23:14:58.367449 1859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "b918721d1d2c42edbdf7437d72970347"
format_stamp: "Formatted at 2026-09-24 23:14:58 on dist-test-slave-m0z6"
server_key: "de92ec7b15e3a0ad37f747f1449eb4ab"
server_key_iv: "03d90e920f6d09dbdef75a9a788b2a01"
server_key_version: "encryptionkey@0"
I20260924 23:14:58.368693 1859 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:14:58.369421 1874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:58.369616 1859 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:58.369701 1859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "b918721d1d2c42edbdf7437d72970347"
format_stamp: "Formatted at 2026-09-24 23:14:58 on dist-test-slave-m0z6"
server_key: "de92ec7b15e3a0ad37f747f1449eb4ab"
server_key_iv: "03d90e920f6d09dbdef75a9a788b2a01"
server_key_version: "encryptionkey@0"
I20260924 23:14:58.369757 1859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:58.393038 1859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:58.395072 1859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:58.395193 1859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:58.399053 1859 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43969
I20260924 23:14:58.399085 1926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43969 every 8 connection(s)
I20260924 23:14:58.399439 1859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:14:58.400332 1927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:58.402575 1927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347: Bootstrap starting.
I20260924 23:14:58.403146 1927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:58.403388 1927 log.cc:824] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:58.404070 1927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347: No bootstrap required, opened a new log
I20260924 23:14:58.405601 1927 raft_consensus.cc:348] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b918721d1d2c42edbdf7437d72970347" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43969 } }
I20260924 23:14:58.405745 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 1859
I20260924 23:14:58.405755 1927 raft_consensus.cc:374] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:58.405793 1927 raft_consensus.cc:729] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b918721d1d2c42edbdf7437d72970347, State: Initialized, Role: FOLLOWER
I20260924 23:14:58.405838 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:14:58.405954 1927 consensus_queue.cc:260] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b918721d1d2c42edbdf7437d72970347" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43969 } }
I20260924 23:14:58.406005 13136 external_mini_cluster.cc:1428] Setting key f4b8c6513fc98a871ddd6ddb6eb49e81
I20260924 23:14:58.406033 1927 raft_consensus.cc:388] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:14:58.406065 1927 raft_consensus.cc:482] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:14:58.406114 1927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:58.406809 1927 raft_consensus.cc:504] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b918721d1d2c42edbdf7437d72970347" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43969 } }
I20260924 23:14:58.406912 1927 leader_election.cc:302] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b918721d1d2c42edbdf7437d72970347; no voters:
I20260924 23:14:58.407104 1927 leader_election.cc:288] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:14:58.407135 1932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:58.407240 1932 raft_consensus.cc:686] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347 [term 1 LEADER]: Becoming Leader. State: Replica: b918721d1d2c42edbdf7437d72970347, State: Running, Role: LEADER
I20260924 23:14:58.407367 1927 sys_catalog.cc:564] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:14:58.407379 1932 consensus_queue.cc:237] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b918721d1d2c42edbdf7437d72970347" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43969 } }
I20260924 23:14:58.407819 1934 sys_catalog.cc:455] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b918721d1d2c42edbdf7437d72970347. Latest consensus state: current_term: 1 leader_uuid: "b918721d1d2c42edbdf7437d72970347" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b918721d1d2c42edbdf7437d72970347" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43969 } } }
I20260924 23:14:58.407903 1934 sys_catalog.cc:458] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:58.408113 1933 sys_catalog.cc:455] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b918721d1d2c42edbdf7437d72970347" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b918721d1d2c42edbdf7437d72970347" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43969 } } }
I20260924 23:14:58.408176 1933 sys_catalog.cc:458] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347 [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:58.408541 1939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:14:58.409093 1939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:14:58.410669 1939 catalog_manager.cc:1347] Generated new cluster ID: 9c8d1e2dc399445eaf9ebb246e5f4108
I20260924 23:14:58.410753 1939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 23:14:58 dist-test-slave-m0z6 krb5kdc[1846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 23:14:58.418069 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:58.406490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:56089 (local address 127.0.0.1:43969)
0924 23:14:58.406998 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:58.407008 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:58.407035 (+ 27us) server_negotiation.cc:407] Connection header received
0924 23:14:58.407135 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:58.407141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:58.407205 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:58.408153 (+ 948us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:58.408263 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:58.409536 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:58.410516 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:58.411144 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:58.411242 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:58.414050 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:58.414057 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:58.414058 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:58.414074 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:58.416220 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:58.416699 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:58.416703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:58.416704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:58.416846 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:58.417059 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:58.417060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:58.417061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:58.417441 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:58.417626 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:58.417714 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 23:14:58.417830 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":423,"threads_started":1}
I20260924 23:14:58.419054 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:14:58.421865 1939 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:14:58.422641 1939 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:14:58.427030 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:58.420578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:38159 (local address 127.0.0.1:43969)
0924 23:14:58.420615 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:58.420617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:58.420741 (+ 124us) server_negotiation.cc:407] Connection header received
0924 23:14:58.420794 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:58.420795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:58.420804 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:58.420874 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:58.421589 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:58.422188 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:58.422974 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:58.423198 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:58.423254 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:58.424599 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:58.424600 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:58.424601 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:14:58.424608 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:58.425893 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:58.426252 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:58.426253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:58.426254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:58.426306 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:14:58.426505 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:14:58.426506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:14:58.426507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:14:58.426613 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:58.426737 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:58.426870 (+ 133us) server_negotiation.cc:299] Negotiation successful
0924 23:14:58.426941 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 23:14:58.435770 1939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b918721d1d2c42edbdf7437d72970347: Generated new TSK 0
I20260924 23:14:58.436021 1939 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:14:58.438551 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 1859
W20260924 23:14:58.442358 1844 connection.cc:535] client connection to 127.0.0.1:43969 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43969): Connection reset by peer (error 104)
2026-09-24T23:14:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1246 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:14:58 dist-test-slave-m0z6 krb5kdc[1958](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:14:58 dist-test-slave-m0z6 krb5kdc[1958](info): set up 2 sockets
Sep 24 23:14:58 dist-test-slave-m0z6 krb5kdc[1958](info): commencing operation
krb5kdc: starting...
W20260924 23:14:59.390775 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:14:59 dist-test-slave-m0z6 krb5kdc[1958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:14:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:14:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 23:14:59.512341 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46091
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:14:59.594084 1971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:14:59.594329 1971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:14:59.594364 1971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:14:59.595736 1971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:14:59.595805 1971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:14:59.595826 1971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:14:59.595841 1971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:14:59.595856 1971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:14:59.597566 1971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46091
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:14:59.597847 1971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:14:59.598074 1971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:14:59.603037 1979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:59.603089 1977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:14:59.603037 1976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:14:59.603323 1971 server_base.cc:1034] running on GCE node
I20260924 23:14:59.603575 1971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:14:59.603883 1971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:14:59.605082 1971 hybrid_clock.cc:648] HybridClock initialized: now 1790291699605059 us; error 39 us; skew 500 ppm
Sep 24 23:14:59 dist-test-slave-m0z6 krb5kdc[1958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291699, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:14:59.612699 1971 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 23:14:59.613255 1971 webserver.cc:460] Webserver started at http://127.0.0.1:35765/ using document root <none> and password file <none>
I20260924 23:14:59.613490 1971 fs_manager.cc:359] Metadata directory not provided
I20260924 23:14:59.613550 1971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:14:59.613688 1971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:14:59.614830 1971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "2f2f2d05cb644a4abe243ca9539b45ce"
format_stamp: "Formatted at 2026-09-24 23:14:59 on dist-test-slave-m0z6"
server_key: "5874356040fb5da480a896e5ff77c9b6"
server_key_iv: "866c0bde5bc21befd64978ded12c0d7d"
server_key_version: "encryptionkey@0"
I20260924 23:14:59.615177 1971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "2f2f2d05cb644a4abe243ca9539b45ce"
format_stamp: "Formatted at 2026-09-24 23:14:59 on dist-test-slave-m0z6"
server_key: "5874356040fb5da480a896e5ff77c9b6"
server_key_iv: "866c0bde5bc21befd64978ded12c0d7d"
server_key_version: "encryptionkey@0"
I20260924 23:14:59.616628 1971 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:14:59.617497 1986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:14:59.617682 1971 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:14:59.617791 1971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "2f2f2d05cb644a4abe243ca9539b45ce"
format_stamp: "Formatted at 2026-09-24 23:14:59 on dist-test-slave-m0z6"
server_key: "5874356040fb5da480a896e5ff77c9b6"
server_key_iv: "866c0bde5bc21befd64978ded12c0d7d"
server_key_version: "encryptionkey@0"
I20260924 23:14:59.617868 1971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:14:59.636675 1971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:14:59.638947 1971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:14:59.639070 1971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:14:59.643205 1971 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46091
I20260924 23:14:59.643297 2038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46091 every 8 connection(s)
I20260924 23:14:59.643632 1971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:14:59.644512 2039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:14:59.646855 2039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce: Bootstrap starting.
I20260924 23:14:59.647468 2039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce: Neither blocks nor log segments found. Creating new log.
I20260924 23:14:59.647536 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 1971
I20260924 23:14:59.647653 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:14:59.647804 2039 log.cc:824] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce: Log is configured to *not* fsync() on all Append() calls
I20260924 23:14:59.647838 13136 external_mini_cluster.cc:1428] Setting key 725e1f4a6ad1778eaa82bccfd55de39c
I20260924 23:14:59.648644 2039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce: No bootstrap required, opened a new log
I20260924 23:14:59.651041 2039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2f2d05cb644a4abe243ca9539b45ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46091 } }
I20260924 23:14:59.651238 2039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:14:59.651270 2039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f2f2d05cb644a4abe243ca9539b45ce, State: Initialized, Role: FOLLOWER
I20260924 23:14:59.651400 2039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2f2d05cb644a4abe243ca9539b45ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46091 } }
I20260924 23:14:59.651469 2039 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:14:59.651508 2039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:14:59.651561 2039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:14:59.652383 2039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2f2d05cb644a4abe243ca9539b45ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46091 } }
I20260924 23:14:59.652500 2039 leader_election.cc:302] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f2f2d05cb644a4abe243ca9539b45ce; no voters:
I20260924 23:14:59.652664 2039 leader_election.cc:288] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:14:59.652704 2044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:14:59.652879 2044 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce [term 1 LEADER]: Becoming Leader. State: Replica: 2f2f2d05cb644a4abe243ca9539b45ce, State: Running, Role: LEADER
I20260924 23:14:59.652920 2039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:14:59.653031 2044 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2f2d05cb644a4abe243ca9539b45ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46091 } }
I20260924 23:14:59.653407 2044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f2f2d05cb644a4abe243ca9539b45ce. Latest consensus state: current_term: 1 leader_uuid: "2f2f2d05cb644a4abe243ca9539b45ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2f2d05cb644a4abe243ca9539b45ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46091 } } }
I20260924 23:14:59.653643 2044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:59.653580 2045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f2f2d05cb644a4abe243ca9539b45ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2f2d05cb644a4abe243ca9539b45ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46091 } } }
I20260924 23:14:59.653959 2043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:59.648487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:49651 (local address 127.0.0.1:46091)
0924 23:14:59.648634 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:59.648644 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:59.648671 (+ 27us) server_negotiation.cc:407] Connection header received
0924 23:14:59.649155 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:59.649162 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:59.649219 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:59.649337 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:59.649562 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:59.650454 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:59.651577 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:59.652191 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:59.652320 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:59.653066 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:59.653070 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:59.653071 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:59.653279 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:59.653320 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:59.653335 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:59.653675 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:59.653724 (+ 49us) server_negotiation.cc:299] Negotiation successful
0924 23:14:59.653755 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":54,"threads_started":1}
I20260924 23:14:59.653981 2045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce [sys.catalog]: This master's current role is: LEADER
I20260924 23:14:59.654141 2053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:14:59.654637 2053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:14:59.655184 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:14:59.656332 2053 catalog_manager.cc:1347] Generated new cluster ID: 69d29c2b1be34733896de4d207a05e9c
I20260924 23:14:59.656385 2053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:14:59.662256 2043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:14:59.658806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:49193 (local address 127.0.0.1:46091)
0924 23:14:59.658852 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 23:14:59.658854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:14:59.658865 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:14:59.658887 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:14:59.658888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:14:59.658898 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:14:59.658961 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:14:59.659142 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:59.659901 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:59.660904 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:14:59.661236 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:14:59.661303 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:14:59.662013 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:14:59.662014 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:14:59.662015 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:14:59.662020 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:14:59.662035 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:14:59.662040 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:14:59.662096 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 23:14:59.662160 (+ 64us) server_negotiation.cc:299] Negotiation successful
0924 23:14:59.662172 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 23:14:59.676620 2053 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:14:59.677181 2053 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:14:59.684101 2053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f2f2d05cb644a4abe243ca9539b45ce: Generated new TSK 0
I20260924 23:14:59.684314 2053 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:14:59.693815 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 1971
W20260924 23:14:59.697628 1956 connection.cc:535] client connection to 127.0.0.1:46091 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46091): Connection reset by peer (error 104)
2026-09-24T23:14:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1254 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:14:59 dist-test-slave-m0z6 krb5kdc[2070](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:14:59 dist-test-slave-m0z6 krb5kdc[2070](info): set up 2 sockets
Sep 24 23:14:59 dist-test-slave-m0z6 krb5kdc[2070](info): commencing operation
krb5kdc: starting...
W20260924 23:15:00.650664 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:15:00 dist-test-slave-m0z6 krb5kdc[2070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:15:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:15:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 23:15:00.775956 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41215
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:15:00.857013 2083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:15:00.857277 2083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:15:00.857311 2083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:15:00.858542 2083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:15:00.858588 2083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:15:00.858608 2083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:15:00.858623 2083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:15:00.858639 2083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:15:00.860220 2083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41215
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:15:00.860476 2083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:15:00.860687 2083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:15:00.865329 2089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:00.865336 2088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:00.865334 2091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:15:00.865648 2083 server_base.cc:1034] running on GCE node
I20260924 23:15:00.865939 2083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:15:00.866222 2083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:15:00.867391 2083 hybrid_clock.cc:648] HybridClock initialized: now 1790291700867370 us; error 30 us; skew 500 ppm
Sep 24 23:15:00 dist-test-slave-m0z6 krb5kdc[2070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291700, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:15:00.870003 2083 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 23:15:00.870517 2083 webserver.cc:460] Webserver started at http://127.0.0.1:44525/ using document root <none> and password file <none>
I20260924 23:15:00.870779 2083 fs_manager.cc:359] Metadata directory not provided
I20260924 23:15:00.870831 2083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:15:00.870978 2083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:15:00.872076 2083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "ee04c7ecad144d86a6be942b40aadb13"
format_stamp: "Formatted at 2026-09-24 23:15:00 on dist-test-slave-m0z6"
server_key: "844a4ce82dc327715346efab6b6d7f11"
server_key_iv: "4f367f7c64898bf2ecacebc62daf5877"
server_key_version: "encryptionkey@0"
I20260924 23:15:00.872422 2083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "ee04c7ecad144d86a6be942b40aadb13"
format_stamp: "Formatted at 2026-09-24 23:15:00 on dist-test-slave-m0z6"
server_key: "844a4ce82dc327715346efab6b6d7f11"
server_key_iv: "4f367f7c64898bf2ecacebc62daf5877"
server_key_version: "encryptionkey@0"
I20260924 23:15:00.873732 2083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:15:00.874395 2098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:15:00.874583 2083 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:15:00.874667 2083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "ee04c7ecad144d86a6be942b40aadb13"
format_stamp: "Formatted at 2026-09-24 23:15:00 on dist-test-slave-m0z6"
server_key: "844a4ce82dc327715346efab6b6d7f11"
server_key_iv: "4f367f7c64898bf2ecacebc62daf5877"
server_key_version: "encryptionkey@0"
I20260924 23:15:00.874759 2083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:15:00.878005 2083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:15:00.879885 2083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:15:00.880018 2083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:15:00.883828 2083 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41215
I20260924 23:15:00.883863 2150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41215 every 8 connection(s)
I20260924 23:15:00.884213 2083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:15:00.885092 2151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:15:00.887498 2151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13: Bootstrap starting.
I20260924 23:15:00.888083 2151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13: Neither blocks nor log segments found. Creating new log.
I20260924 23:15:00.888326 2151 log.cc:824] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13: Log is configured to *not* fsync() on all Append() calls
I20260924 23:15:00.889052 2151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13: No bootstrap required, opened a new log
I20260924 23:15:00.890625 2151 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee04c7ecad144d86a6be942b40aadb13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41215 } }
I20260924 23:15:00.890806 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 2083
I20260924 23:15:00.890832 2151 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:15:00.890861 2151 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee04c7ecad144d86a6be942b40aadb13, State: Initialized, Role: FOLLOWER
I20260924 23:15:00.890910 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:15:00.891026 2151 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee04c7ecad144d86a6be942b40aadb13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41215 } }
I20260924 23:15:00.891103 2151 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:15:00.891110 13136 external_mini_cluster.cc:1428] Setting key ae6066c207e90d5b796cc5814147553b
I20260924 23:15:00.891134 2151 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:15:00.891186 2151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:15:00.892149 2151 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee04c7ecad144d86a6be942b40aadb13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41215 } }
I20260924 23:15:00.892251 2151 leader_election.cc:302] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee04c7ecad144d86a6be942b40aadb13; no voters:
I20260924 23:15:00.892658 2151 leader_election.cc:288] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:15:00.892704 2156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:15:00.892877 2156 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13 [term 1 LEADER]: Becoming Leader. State: Replica: ee04c7ecad144d86a6be942b40aadb13, State: Running, Role: LEADER
I20260924 23:15:00.893018 2151 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:15:00.893270 2155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:00.891645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:34183 (local address 127.0.0.1:41215)
0924 23:15:00.891839 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:00.891848 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:00.891875 (+ 27us) server_negotiation.cc:407] Connection header received
0924 23:15:00.891973 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:00.891979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:00.892003 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:00.892121 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:00.892747 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:00.892751 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:00.892751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:15:00.892756 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:00.892822 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:15:00.892839 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:00.892913 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:00.893032 (+ 119us) server_negotiation.cc:299] Negotiation successful
0924 23:15:00.893091 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
I20260924 23:15:00.893007 2156 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee04c7ecad144d86a6be942b40aadb13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41215 } }
I20260924 23:15:00.894333 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:15:00.894800 2157 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee04c7ecad144d86a6be942b40aadb13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee04c7ecad144d86a6be942b40aadb13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41215 } } }
I20260924 23:15:00.895062 2157 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13 [sys.catalog]: This master's current role is: LEADER
I20260924 23:15:00.894842 2158 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee04c7ecad144d86a6be942b40aadb13. Latest consensus state: current_term: 1 leader_uuid: "ee04c7ecad144d86a6be942b40aadb13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee04c7ecad144d86a6be942b40aadb13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41215 } } }
I20260924 23:15:00.895279 2158 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13 [sys.catalog]: This master's current role is: LEADER
W20260924 23:15:00.895403 2170 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:15:00.895447 2170 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:15:00.895496 2172 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:15:00.895705 2172 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:15:00.896997 2155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:00.895886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:48141 (local address 127.0.0.1:41215)
0924 23:15:00.895917 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:00.895919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:00.896089 (+ 170us) server_negotiation.cc:407] Connection header received
0924 23:15:00.896109 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:00.896110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:00.896114 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:00.896163 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:00.896727 (+ 564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:00.896729 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:00.896729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:15:00.896734 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:00.896750 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:15:00.896754 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:00.896811 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:00.896861 (+ 50us) server_negotiation.cc:299] Negotiation successful
0924 23:15:00.896876 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260924 23:15:00.897378 2172 catalog_manager.cc:1347] Generated new cluster ID: fd78db63b3aa446cbc28f8325be57886
I20260924 23:15:00.897426 2172 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:15:00.923709 2172 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:15:00.923964 2172 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:15:00.930598 2172 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee04c7ecad144d86a6be942b40aadb13: Generated new TSK 0
I20260924 23:15:00.930860 2172 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:15:00.935814 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 2083
2026-09-24T23:15:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1242 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:15:00 dist-test-slave-m0z6 krb5kdc[2182](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:15:00 dist-test-slave-m0z6 krb5kdc[2182](info): set up 2 sockets
Sep 24 23:15:00 dist-test-slave-m0z6 krb5kdc[2182](info): commencing operation
krb5kdc: starting...
W20260924 23:15:01.894543 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:15:01 dist-test-slave-m0z6 krb5kdc[2182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:15:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:15:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:15:02.014341 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:39063
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:36757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:39063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:15:02.097010 2195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:15:02.097249 2195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:15:02.097281 2195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:15:02.098515 2195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:15:02.098567 2195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:15:02.098584 2195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:15:02.098600 2195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:15:02.098615 2195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:15:02.100194 2195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:36757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:39063
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:39063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:15:02.100479 2195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:15:02.100719 2195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:15:02.105839 2203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:02.105855 2200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:15:02.106009 2195 server_base.cc:1034] running on GCE node
W20260924 23:15:02.105844 2201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:15:02.106343 2195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:15:02.106658 2195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:15:02.107831 2195 hybrid_clock.cc:648] HybridClock initialized: now 1790291702107821 us; error 31 us; skew 500 ppm
Sep 24 23:15:02 dist-test-slave-m0z6 krb5kdc[2182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291702, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:15:02.110340 2195 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:15:02.110862 2195 webserver.cc:460] Webserver started at http://127.12.212.62:38145/ using document root <none> and password file <none>
I20260924 23:15:02.111099 2195 fs_manager.cc:359] Metadata directory not provided
I20260924 23:15:02.111153 2195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:15:02.111279 2195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:15:02.112352 2195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "69600d33dd954ecabeb3f38e4a46624e"
format_stamp: "Formatted at 2026-09-24 23:15:02 on dist-test-slave-m0z6"
server_key: "9a6d3831a87079c7136ad43dbd5ff36a"
server_key_iv: "ac599dfe651d70b8b0b4a165e40f4566"
server_key_version: "encryptionkey@0"
I20260924 23:15:02.112684 2195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "69600d33dd954ecabeb3f38e4a46624e"
format_stamp: "Formatted at 2026-09-24 23:15:02 on dist-test-slave-m0z6"
server_key: "9a6d3831a87079c7136ad43dbd5ff36a"
server_key_iv: "ac599dfe651d70b8b0b4a165e40f4566"
server_key_version: "encryptionkey@0"
I20260924 23:15:02.113967 2195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:15:02.114768 2210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:15:02.114923 2195 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 23:15:02.115005 2195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "69600d33dd954ecabeb3f38e4a46624e"
format_stamp: "Formatted at 2026-09-24 23:15:02 on dist-test-slave-m0z6"
server_key: "9a6d3831a87079c7136ad43dbd5ff36a"
server_key_iv: "ac599dfe651d70b8b0b4a165e40f4566"
server_key_version: "encryptionkey@0"
I20260924 23:15:02.115113 2195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:15:02.127265 2195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:15:02.129801 2195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:15:02.129966 2195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:15:02.134119 2195 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:39063
I20260924 23:15:02.134200 2262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:39063 every 8 connection(s)
I20260924 23:15:02.134513 2195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:15:02.135447 2263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:15:02.137827 2263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e: Bootstrap starting.
I20260924 23:15:02.138442 2263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e: Neither blocks nor log segments found. Creating new log.
I20260924 23:15:02.138716 2263 log.cc:824] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e: Log is configured to *not* fsync() on all Append() calls
I20260924 23:15:02.139374 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 2195
I20260924 23:15:02.139470 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:15:02.139492 2263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e: No bootstrap required, opened a new log
I20260924 23:15:02.139644 13136 external_mini_cluster.cc:1428] Setting key b047121b825a53ed3940fe179775d940
I20260924 23:15:02.141669 2263 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69600d33dd954ecabeb3f38e4a46624e" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39063 } }
I20260924 23:15:02.141901 2263 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:15:02.141956 2263 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69600d33dd954ecabeb3f38e4a46624e, State: Initialized, Role: FOLLOWER
I20260924 23:15:02.142112 2263 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69600d33dd954ecabeb3f38e4a46624e" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39063 } }
I20260924 23:15:02.142225 2263 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:15:02.142298 2263 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:15:02.142351 2263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:15:02.143147 2263 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69600d33dd954ecabeb3f38e4a46624e" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39063 } }
I20260924 23:15:02.143260 2263 leader_election.cc:302] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69600d33dd954ecabeb3f38e4a46624e; no voters:
I20260924 23:15:02.143426 2263 leader_election.cc:288] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:15:02.143584 2269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:15:02.143671 2263 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:15:02.143766 2269 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e [term 1 LEADER]: Becoming Leader. State: Replica: 69600d33dd954ecabeb3f38e4a46624e, State: Running, Role: LEADER
I20260924 23:15:02.143913 2269 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69600d33dd954ecabeb3f38e4a46624e" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39063 } }
I20260924 23:15:02.144297 2269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69600d33dd954ecabeb3f38e4a46624e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69600d33dd954ecabeb3f38e4a46624e" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39063 } } }
I20260924 23:15:02.144409 2269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e [sys.catalog]: This master's current role is: LEADER
I20260924 23:15:02.144575 2269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69600d33dd954ecabeb3f38e4a46624e. Latest consensus state: current_term: 1 leader_uuid: "69600d33dd954ecabeb3f38e4a46624e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69600d33dd954ecabeb3f38e4a46624e" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 39063 } } }
I20260924 23:15:02.144683 2269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e [sys.catalog]: This master's current role is: LEADER
W20260924 23:15:02.145483 2283 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:15:02.145579 2283 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:15:02.145939 2277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:15:02.146170 2277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:15:02.147907 2277 catalog_manager.cc:1347] Generated new cluster ID: ca9a0ab45f3b4ee294014c0cdeeeb2e1
I20260924 23:15:02.147955 2277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 23:15:02 dist-test-slave-m0z6 krb5kdc[2182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:15:02.152942 2267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:02.140181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45019 (local address 127.12.212.62:39063)
0924 23:15:02.140366 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:02.140376 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:02.140410 (+ 34us) server_negotiation.cc:407] Connection header received
0924 23:15:02.140824 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:02.140833 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:02.140916 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:02.141070 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:02.141875 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:02.143555 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:02.144681 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:02.145320 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:02.146398 (+ 1078us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:02.149252 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:02.149256 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:02.149257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:02.149270 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:02.151217 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:02.151675 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:02.151678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:02.151679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:02.151780 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:02.152017 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:02.152017 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:02.152018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:02.152332 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:02.152490 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:02.152627 (+ 137us) server_negotiation.cc:299] Negotiation successful
0924 23:15:02.152728 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260924 23:15:02.154284 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:15:02.159931 2277 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:15:02.160650 2277 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:15:02.165158 2267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:02.159111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39397 (local address 127.12.212.62:39063)
0924 23:15:02.159152 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:02.159154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:02.159323 (+ 169us) server_negotiation.cc:407] Connection header received
0924 23:15:02.159392 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:02.159393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:02.159405 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:02.159453 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:02.160137 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:02.160620 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:02.161327 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:02.161533 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:02.161588 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:02.162780 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:02.162781 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:02.162781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:02.162789 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:02.164099 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:02.164448 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:02.164450 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:02.164450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:02.164501 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:02.164690 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:02.164691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:02.164691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:02.164795 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:02.164895 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:02.165007 (+ 112us) server_negotiation.cc:299] Negotiation successful
0924 23:15:02.165069 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 23:15:02.171692 2277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69600d33dd954ecabeb3f38e4a46624e: Generated new TSK 0
I20260924 23:15:02.171945 2277 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:15:02.175770 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 2195
W20260924 23:15:02.179668 2180 connection.cc:535] client connection to 127.12.212.62:39063 recv error: Network error: failed to read from TLS socket (remote: 127.12.212.62:39063): Connection reset by peer (error 104)
2026-09-24T23:15:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1239 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:15:02 dist-test-slave-m0z6 krb5kdc[2294](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:15:02 dist-test-slave-m0z6 krb5kdc[2294](info): set up 2 sockets
Sep 24 23:15:02 dist-test-slave-m0z6 krb5kdc[2294](info): commencing operation
krb5kdc: starting...
W20260924 23:15:03.122601 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:15:03 dist-test-slave-m0z6 krb5kdc[2294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:15:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:15:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:15:03.241489 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:35941
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:38695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:35941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:15:03.322840 2307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:15:03.323078 2307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:15:03.323103 2307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:15:03.324364 2307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:15:03.324414 2307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:15:03.324426 2307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:15:03.324438 2307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:15:03.324450 2307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:15:03.326020 2307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:38695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:35941
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:35941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:15:03.326244 2307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:15:03.326447 2307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:15:03.331521 2313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:03.331502 2312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:15:03.331543 2307 server_base.cc:1034] running on GCE node
W20260924 23:15:03.331502 2315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:15:03.331992 2307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:15:03.332302 2307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:15:03.333474 2307 hybrid_clock.cc:648] HybridClock initialized: now 1790291703333454 us; error 38 us; skew 500 ppm
Sep 24 23:15:03 dist-test-slave-m0z6 krb5kdc[2294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291703, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:15:03.336166 2307 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:15:03.336633 2307 webserver.cc:460] Webserver started at http://127.12.212.62:35579/ using document root <none> and password file <none>
I20260924 23:15:03.336839 2307 fs_manager.cc:359] Metadata directory not provided
I20260924 23:15:03.336889 2307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:15:03.337023 2307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:15:03.338104 2307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "fc83e401297b4e7396e7523187b5f24f"
format_stamp: "Formatted at 2026-09-24 23:15:03 on dist-test-slave-m0z6"
server_key: "76fe57d7217170491e3438c138dde46c"
server_key_iv: "477e86ec1f04321c9830dbeeec2afaca"
server_key_version: "encryptionkey@0"
I20260924 23:15:03.338443 2307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "fc83e401297b4e7396e7523187b5f24f"
format_stamp: "Formatted at 2026-09-24 23:15:03 on dist-test-slave-m0z6"
server_key: "76fe57d7217170491e3438c138dde46c"
server_key_iv: "477e86ec1f04321c9830dbeeec2afaca"
server_key_version: "encryptionkey@0"
I20260924 23:15:03.339804 2307 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:15:03.340655 2322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:15:03.340840 2307 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:15:03.340934 2307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "fc83e401297b4e7396e7523187b5f24f"
format_stamp: "Formatted at 2026-09-24 23:15:03 on dist-test-slave-m0z6"
server_key: "76fe57d7217170491e3438c138dde46c"
server_key_iv: "477e86ec1f04321c9830dbeeec2afaca"
server_key_version: "encryptionkey@0"
I20260924 23:15:03.341002 2307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:15:03.357092 2307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:15:03.359525 2307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:15:03.359678 2307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:15:03.363432 2307 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:35941
I20260924 23:15:03.363819 2307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:15:03.363804 2374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:35941 every 8 connection(s)
I20260924 23:15:03.364648 2375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:15:03.366683 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 2307
I20260924 23:15:03.366842 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:15:03.367050 13136 external_mini_cluster.cc:1428] Setting key 5cd47dfd0b5b5a63341e12eb12f7ce46
I20260924 23:15:03.367313 2375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f: Bootstrap starting.
I20260924 23:15:03.368036 2375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f: Neither blocks nor log segments found. Creating new log.
I20260924 23:15:03.368319 2375 log.cc:824] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f: Log is configured to *not* fsync() on all Append() calls
I20260924 23:15:03.369064 2375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f: No bootstrap required, opened a new log
I20260924 23:15:03.371304 2375 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc83e401297b4e7396e7523187b5f24f" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 35941 } }
I20260924 23:15:03.371483 2375 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:15:03.371533 2375 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc83e401297b4e7396e7523187b5f24f, State: Initialized, Role: FOLLOWER
I20260924 23:15:03.371729 2375 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc83e401297b4e7396e7523187b5f24f" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 35941 } }
I20260924 23:15:03.371809 2375 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:15:03.371833 2375 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:15:03.371881 2375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:15:03.372483 2375 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc83e401297b4e7396e7523187b5f24f" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 35941 } }
I20260924 23:15:03.372591 2375 leader_election.cc:302] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc83e401297b4e7396e7523187b5f24f; no voters:
I20260924 23:15:03.372769 2375 leader_election.cc:288] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:15:03.372804 2380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:15:03.372969 2380 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f [term 1 LEADER]: Becoming Leader. State: Replica: fc83e401297b4e7396e7523187b5f24f, State: Running, Role: LEADER
I20260924 23:15:03.373008 2378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:03.367630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40557 (local address 127.12.212.62:35941)
0924 23:15:03.367823 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:03.367833 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:03.367864 (+ 31us) server_negotiation.cc:407] Connection header received
0924 23:15:03.368355 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:03.368365 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:03.368446 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:03.368598 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:03.368780 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:03.369739 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:03.370884 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:03.371331 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:03.371457 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:03.372175 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:03.372179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:03.372180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:15:03.372347 (+ 167us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:03.372381 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:15:03.372393 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:03.372496 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:03.372591 (+ 95us) server_negotiation.cc:299] Negotiation successful
0924 23:15:03.372713 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":90,"threads_started":1}
I20260924 23:15:03.373045 2375 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:15:03.373121 2380 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc83e401297b4e7396e7523187b5f24f" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 35941 } }
I20260924 23:15:03.373490 2381 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc83e401297b4e7396e7523187b5f24f. Latest consensus state: current_term: 1 leader_uuid: "fc83e401297b4e7396e7523187b5f24f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc83e401297b4e7396e7523187b5f24f" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 35941 } } }
I20260924 23:15:03.373512 2382 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc83e401297b4e7396e7523187b5f24f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc83e401297b4e7396e7523187b5f24f" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 35941 } } }
I20260924 23:15:03.373657 2381 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f [sys.catalog]: This master's current role is: LEADER
I20260924 23:15:03.373667 2382 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f [sys.catalog]: This master's current role is: LEADER
I20260924 23:15:03.374099 2389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:15:03.374604 2389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:15:03.375721 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:15:03.376204 2389 catalog_manager.cc:1347] Generated new cluster ID: a20f2e2e5f624a3ea753b855cba9e8ed
I20260924 23:15:03.376250 2389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:15:03.381673 2378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:03.377043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51131 (local address 127.12.212.62:35941)
0924 23:15:03.377080 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:03.377082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:03.379026 (+ 1944us) server_negotiation.cc:407] Connection header received
0924 23:15:03.379100 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:03.379101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:03.379112 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:03.379161 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:03.379296 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:03.379800 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:03.380514 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:03.380729 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:03.380783 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:03.381437 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:03.381438 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:03.381439 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:15:03.381444 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:03.381460 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:15:03.381465 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:03.381508 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:03.381577 (+ 69us) server_negotiation.cc:299] Negotiation successful
0924 23:15:03.381587 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 23:15:03.416719 2389 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:15:03.417343 2389 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:15:03.420511 2389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc83e401297b4e7396e7523187b5f24f: Generated new TSK 0
I20260924 23:15:03.420712 2389 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:15:03.425566 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 2307
W20260924 23:15:03.429631 2292 connection.cc:535] client connection to 127.12.212.62:35941 recv error: Network error: failed to read from TLS socket (remote: 127.12.212.62:35941): Connection reset by peer (error 104)
2026-09-24T23:15:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1249 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:15:03 dist-test-slave-m0z6 krb5kdc[2406](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:15:03 dist-test-slave-m0z6 krb5kdc[2406](info): set up 2 sockets
Sep 24 23:15:03 dist-test-slave-m0z6 krb5kdc[2406](info): commencing operation
krb5kdc: starting...
W20260924 23:15:04.394629 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.952s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:15:04 dist-test-slave-m0z6 krb5kdc[2406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:15:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:15:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:15:04.515563 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:33563
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:44439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:33563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:15:04.596612 2419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:15:04.596843 2419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:15:04.596866 2419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:15:04.598076 2419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:15:04.598133 2419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:15:04.598146 2419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:15:04.598158 2419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:15:04.598169 2419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:15:04.599829 2419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:44439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:33563
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:33563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:15:04.600106 2419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:15:04.600332 2419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:15:04.605207 2424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:15:04.605221 2419 server_base.cc:1034] running on GCE node
W20260924 23:15:04.605227 2427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:04.605232 2425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:15:04.605650 2419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:15:04.605945 2419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:15:04.607106 2419 hybrid_clock.cc:648] HybridClock initialized: now 1790291704607082 us; error 46 us; skew 500 ppm
Sep 24 23:15:04 dist-test-slave-m0z6 krb5kdc[2406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291704, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:15:04.609596 2419 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:15:04.610075 2419 webserver.cc:460] Webserver started at http://127.12.212.62:43839/ using document root <none> and password file <none>
I20260924 23:15:04.610294 2419 fs_manager.cc:359] Metadata directory not provided
I20260924 23:15:04.610342 2419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:15:04.610471 2419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:15:04.611589 2419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "522729bb0a5346458cd60710e6716c4a"
format_stamp: "Formatted at 2026-09-24 23:15:04 on dist-test-slave-m0z6"
server_key: "5ebd693a57196e6ffaee27beffa38e0c"
server_key_iv: "2ca9e8453cdabb4c1b9c364923d45352"
server_key_version: "encryptionkey@0"
I20260924 23:15:04.611930 2419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "522729bb0a5346458cd60710e6716c4a"
format_stamp: "Formatted at 2026-09-24 23:15:04 on dist-test-slave-m0z6"
server_key: "5ebd693a57196e6ffaee27beffa38e0c"
server_key_iv: "2ca9e8453cdabb4c1b9c364923d45352"
server_key_version: "encryptionkey@0"
I20260924 23:15:04.613241 2419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:15:04.614055 2434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:15:04.614241 2419 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:15:04.614326 2419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "522729bb0a5346458cd60710e6716c4a"
format_stamp: "Formatted at 2026-09-24 23:15:04 on dist-test-slave-m0z6"
server_key: "5ebd693a57196e6ffaee27beffa38e0c"
server_key_iv: "2ca9e8453cdabb4c1b9c364923d45352"
server_key_version: "encryptionkey@0"
I20260924 23:15:04.614384 2419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:15:04.617755 2419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:15:04.635262 2419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:15:04.635396 2419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:15:04.639267 2419 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:33563
I20260924 23:15:04.639299 2486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:33563 every 8 connection(s)
I20260924 23:15:04.639636 2419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:15:04.640483 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 2419
I20260924 23:15:04.640626 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:15:04.640821 13136 external_mini_cluster.cc:1428] Setting key 749743107d334445d0c40d94d589a426
I20260924 23:15:04.640820 2487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:15:04.643584 2490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:04.641387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52409 (local address 127.12.212.62:33563)
0924 23:15:04.641566 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:04.641576 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:04.641606 (+ 30us) server_negotiation.cc:407] Connection header received
0924 23:15:04.642247 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:04.642255 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:04.642285 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:04.642438 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:04.643154 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:04.643157 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:04.643157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:15:04.643161 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:04.643200 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:15:04.643211 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:04.643269 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:04.643345 (+ 76us) server_negotiation.cc:299] Negotiation successful
0924 23:15:04.643383 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260924 23:15:04.644174 2487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a: Bootstrap starting.
I20260924 23:15:04.644788 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:15:04.644940 2487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a: Neither blocks nor log segments found. Creating new log.
I20260924 23:15:04.645215 2487 log.cc:824] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a: Log is configured to *not* fsync() on all Append() calls
I20260924 23:15:04.646034 2487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a: No bootstrap required, opened a new log
I20260924 23:15:04.648025 2487 raft_consensus.cc:348] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "522729bb0a5346458cd60710e6716c4a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 33563 } }
I20260924 23:15:04.648185 2487 raft_consensus.cc:374] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:15:04.648208 2487 raft_consensus.cc:729] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522729bb0a5346458cd60710e6716c4a, State: Initialized, Role: FOLLOWER
I20260924 23:15:04.648321 2487 consensus_queue.cc:260] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "522729bb0a5346458cd60710e6716c4a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 33563 } }
I20260924 23:15:04.648458 2487 raft_consensus.cc:388] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:15:04.648533 2487 raft_consensus.cc:482] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:15:04.648594 2487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:15:04.649223 2487 raft_consensus.cc:504] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "522729bb0a5346458cd60710e6716c4a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 33563 } }
I20260924 23:15:04.649325 2487 leader_election.cc:302] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 522729bb0a5346458cd60710e6716c4a; no voters:
I20260924 23:15:04.649500 2487 leader_election.cc:288] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:15:04.649619 2497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:15:04.649742 2487 sys_catalog.cc:564] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:15:04.649809 2497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a [term 1 LEADER]: Becoming Leader. State: Replica: 522729bb0a5346458cd60710e6716c4a, State: Running, Role: LEADER
I20260924 23:15:04.649910 2497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "522729bb0a5346458cd60710e6716c4a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 33563 } }
I20260924 23:15:04.650411 2499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 522729bb0a5346458cd60710e6716c4a. Latest consensus state: current_term: 1 leader_uuid: "522729bb0a5346458cd60710e6716c4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "522729bb0a5346458cd60710e6716c4a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 33563 } } }
I20260924 23:15:04.650378 2498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "522729bb0a5346458cd60710e6716c4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "522729bb0a5346458cd60710e6716c4a" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 33563 } } }
I20260924 23:15:04.650499 2499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a [sys.catalog]: This master's current role is: LEADER
I20260924 23:15:04.650518 2498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a [sys.catalog]: This master's current role is: LEADER
I20260924 23:15:04.651149 2506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:15:04.651717 2490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:04.646315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37933 (local address 127.12.212.62:33563)
0924 23:15:04.646366 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:04.646369 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:04.650498 (+ 4129us) server_negotiation.cc:407] Connection header received
0924 23:15:04.650644 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:04.650645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:04.650651 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:04.650765 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:04.651484 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:04.651486 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:04.651486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:15:04.651490 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:04.651507 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:15:04.651511 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:04.651569 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:04.651640 (+ 71us) server_negotiation.cc:299] Negotiation successful
0924 23:15:04.651655 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 23:15:04.651909 2506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:15:04.653723 2506 catalog_manager.cc:1347] Generated new cluster ID: 4020d72af495430488b25308ef18bc31
I20260924 23:15:04.653786 2506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:15:04.661222 2506 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:15:04.661414 2506 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:15:04.669405 2506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 522729bb0a5346458cd60710e6716c4a: Generated new TSK 0
I20260924 23:15:04.669643 2506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:15:04.676882 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 2419
2026-09-24T23:15:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1251 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:15:04 dist-test-slave-m0z6 krb5kdc[2518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:15:04 dist-test-slave-m0z6 krb5kdc[2518](info): set up 2 sockets
Sep 24 23:15:04 dist-test-slave-m0z6 krb5kdc[2518](info): commencing operation
krb5kdc: starting...
W20260924 23:15:05.634661 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:15:05 dist-test-slave-m0z6 krb5kdc[2518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:15:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:15:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:15:05.751586 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:46399
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:45227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:46399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:15:05.834676 2531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:15:05.834968 2531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:15:05.835004 2531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:15:05.836326 2531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:15:05.836380 2531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:15:05.836398 2531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:15:05.836416 2531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:15:05.836431 2531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:15:05.838119 2531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:45227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:46399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:46399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:15:05.838410 2531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:15:05.838645 2531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:15:05.843478 2539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:05.843636 2537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:05.843498 2536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:15:05.843770 2531 server_base.cc:1034] running on GCE node
I20260924 23:15:05.844062 2531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:15:05.844353 2531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:15:05.845511 2531 hybrid_clock.cc:648] HybridClock initialized: now 1790291705845490 us; error 37 us; skew 500 ppm
Sep 24 23:15:05 dist-test-slave-m0z6 krb5kdc[2518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291705, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:15:05.848006 2531 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:15:05.848433 2531 webserver.cc:460] Webserver started at http://127.12.212.62:42963/ using document root <none> and password file <none>
I20260924 23:15:05.848641 2531 fs_manager.cc:359] Metadata directory not provided
I20260924 23:15:05.848686 2531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:15:05.848829 2531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:15:05.849856 2531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "7b1dc52080a64492b974557c51fd59cf"
format_stamp: "Formatted at 2026-09-24 23:15:05 on dist-test-slave-m0z6"
server_key: "0993cb12d76b526c8b7b8e5229eb23fa"
server_key_iv: "fce50e6ab15fb807e3aac18ea910b09b"
server_key_version: "encryptionkey@0"
I20260924 23:15:05.850178 2531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "7b1dc52080a64492b974557c51fd59cf"
format_stamp: "Formatted at 2026-09-24 23:15:05 on dist-test-slave-m0z6"
server_key: "0993cb12d76b526c8b7b8e5229eb23fa"
server_key_iv: "fce50e6ab15fb807e3aac18ea910b09b"
server_key_version: "encryptionkey@0"
I20260924 23:15:05.851491 2531 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:15:05.852262 2546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:15:05.852494 2531 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:15:05.852581 2531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "7b1dc52080a64492b974557c51fd59cf"
format_stamp: "Formatted at 2026-09-24 23:15:05 on dist-test-slave-m0z6"
server_key: "0993cb12d76b526c8b7b8e5229eb23fa"
server_key_iv: "fce50e6ab15fb807e3aac18ea910b09b"
server_key_version: "encryptionkey@0"
I20260924 23:15:05.852648 2531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:15:05.860313 2531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:15:05.862387 2531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:15:05.862529 2531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:15:05.866190 2531 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:46399
I20260924 23:15:05.866276 2598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:46399 every 8 connection(s)
I20260924 23:15:05.866590 2531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:15:05.867570 2599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:15:05.869961 2599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf: Bootstrap starting.
I20260924 23:15:05.870555 2599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf: Neither blocks nor log segments found. Creating new log.
I20260924 23:15:05.870872 2599 log.cc:824] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf: Log is configured to *not* fsync() on all Append() calls
I20260924 23:15:05.871639 2599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf: No bootstrap required, opened a new log
I20260924 23:15:05.873345 2599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b1dc52080a64492b974557c51fd59cf" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46399 } }
I20260924 23:15:05.873497 2599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:15:05.873530 2599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b1dc52080a64492b974557c51fd59cf, State: Initialized, Role: FOLLOWER
I20260924 23:15:05.873669 2599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b1dc52080a64492b974557c51fd59cf" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46399 } }
I20260924 23:15:05.873734 2599 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:15:05.873764 2599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:15:05.873821 2599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:15:05.874750 2599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b1dc52080a64492b974557c51fd59cf" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46399 } }
I20260924 23:15:05.874871 2599 leader_election.cc:302] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b1dc52080a64492b974557c51fd59cf; no voters:
I20260924 23:15:05.875037 2599 leader_election.cc:288] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:15:05.875075 2602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:15:05.875231 2602 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf [term 1 LEADER]: Becoming Leader. State: Replica: 7b1dc52080a64492b974557c51fd59cf, State: Running, Role: LEADER
I20260924 23:15:05.875294 2599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:15:05.875387 2602 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b1dc52080a64492b974557c51fd59cf" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46399 } }
I20260924 23:15:05.875742 2603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b1dc52080a64492b974557c51fd59cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b1dc52080a64492b974557c51fd59cf" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46399 } } }
I20260924 23:15:05.875752 2604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b1dc52080a64492b974557c51fd59cf. Latest consensus state: current_term: 1 leader_uuid: "7b1dc52080a64492b974557c51fd59cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b1dc52080a64492b974557c51fd59cf" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 46399 } } }
I20260924 23:15:05.875872 2603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf [sys.catalog]: This master's current role is: LEADER
I20260924 23:15:05.875886 2604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf [sys.catalog]: This master's current role is: LEADER
I20260924 23:15:05.876526 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 2531
I20260924 23:15:05.876632 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:15:05.876840 13136 external_mini_cluster.cc:1428] Setting key 23b9e138fd417846a151a47803c109d0
W20260924 23:15:05.877094 2618 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:15:05.877149 2618 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:15:05.877199 2610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:15:05.877364 2610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:15:05.880776 2610 catalog_manager.cc:1347] Generated new cluster ID: c8752b8db583445eb4d097f7a38e1b77
I20260924 23:15:05.880860 2610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 23:15:05 dist-test-slave-m0z6 krb5kdc[2518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:15:05.888204 2621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:05.877510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41713 (local address 127.12.212.62:46399)
0924 23:15:05.877685 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:05.877695 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:05.877723 (+ 28us) server_negotiation.cc:407] Connection header received
0924 23:15:05.877963 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:05.877969 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:05.878027 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:05.878186 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:05.879003 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:05.880582 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:05.881530 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:05.881995 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:05.882107 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:05.884723 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:05.884729 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:05.884730 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:05.884742 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:05.886384 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:05.886764 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:05.886770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:05.886770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:05.886886 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:05.887096 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:05.887096 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:05.887097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:05.887370 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:05.887531 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:05.887710 (+ 179us) server_negotiation.cc:299] Negotiation successful
0924 23:15:05.887983 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
I20260924 23:15:05.888499 2610 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:15:05.889225 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:15:05.889241 2610 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:15:05.894922 2610 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b1dc52080a64492b974557c51fd59cf: Generated new TSK 0
I20260924 23:15:05.895160 2610 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:15:05.897296 2621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:05.890584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47351 (local address 127.12.212.62:46399)
0924 23:15:05.890673 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:05.890675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:05.890748 (+ 73us) server_negotiation.cc:407] Connection header received
0924 23:15:05.890815 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:05.890816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:05.890838 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:05.890882 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:05.891459 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:05.891973 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:05.892860 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:05.893104 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:05.893165 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:05.894458 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:05.894460 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:05.894460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:05.894468 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:05.895931 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:05.896404 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:05.896405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:05.896406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:05.896519 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:05.896777 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:05.896778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:05.896779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:05.896928 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:05.897017 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:05.897150 (+ 133us) server_negotiation.cc:299] Negotiation successful
0924 23:15:05.897226 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260924 23:15:05.900303 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 2531
2026-09-24T23:15:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1223 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:15:05 dist-test-slave-m0z6 krb5kdc[2630](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:15:05 dist-test-slave-m0z6 krb5kdc[2630](info): set up 2 sockets
Sep 24 23:15:05 dist-test-slave-m0z6 krb5kdc[2630](info): commencing operation
krb5kdc: starting...
W20260924 23:15:06.866617 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:15:06 dist-test-slave-m0z6 krb5kdc[2630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:15:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:15:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:15:06.987241 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:42845
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:46227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:42845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:15:07.069512 2643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:15:07.069746 2643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:15:07.069772 2643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:15:07.071126 2643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:15:07.071182 2643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:15:07.071197 2643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:15:07.071209 2643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:15:07.071220 2643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:15:07.072829 2643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:46227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:42845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:42845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:15:07.073096 2643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:15:07.073364 2643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:15:07.078473 2649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:15:07.078486 2643 server_base.cc:1034] running on GCE node
W20260924 23:15:07.078467 2648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:07.078467 2651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:15:07.079012 2643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:15:07.079341 2643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:15:07.080520 2643 hybrid_clock.cc:648] HybridClock initialized: now 1790291707080488 us; error 49 us; skew 500 ppm
Sep 24 23:15:07 dist-test-slave-m0z6 krb5kdc[2630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291707, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:15:07.083326 2643 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:15:07.083815 2643 webserver.cc:460] Webserver started at http://127.12.212.62:41355/ using document root <none> and password file <none>
I20260924 23:15:07.084069 2643 fs_manager.cc:359] Metadata directory not provided
I20260924 23:15:07.084124 2643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:15:07.084261 2643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:15:07.085427 2643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "4671a395739f4c9cb7316315ce5645d7"
format_stamp: "Formatted at 2026-09-24 23:15:07 on dist-test-slave-m0z6"
server_key: "12ff781549bf1a421d12993a882056da"
server_key_iv: "9f8bcea74fdbfa95f713da345e1fd425"
server_key_version: "encryptionkey@0"
I20260924 23:15:07.085791 2643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "4671a395739f4c9cb7316315ce5645d7"
format_stamp: "Formatted at 2026-09-24 23:15:07 on dist-test-slave-m0z6"
server_key: "12ff781549bf1a421d12993a882056da"
server_key_iv: "9f8bcea74fdbfa95f713da345e1fd425"
server_key_version: "encryptionkey@0"
I20260924 23:15:07.087363 2643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:15:07.088388 2658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:15:07.088634 2643 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:15:07.088734 2643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "4671a395739f4c9cb7316315ce5645d7"
format_stamp: "Formatted at 2026-09-24 23:15:07 on dist-test-slave-m0z6"
server_key: "12ff781549bf1a421d12993a882056da"
server_key_iv: "9f8bcea74fdbfa95f713da345e1fd425"
server_key_version: "encryptionkey@0"
I20260924 23:15:07.088843 2643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:15:07.124081 2643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:15:07.126442 2643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:15:07.126600 2643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:15:07.130402 2643 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:42845
I20260924 23:15:07.130475 2710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:42845 every 8 connection(s)
I20260924 23:15:07.130843 2643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:15:07.131798 2711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:15:07.132452 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 2643
I20260924 23:15:07.132591 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:15:07.132776 13136 external_mini_cluster.cc:1428] Setting key 38d5523f639530683738b310a20a7cf0
I20260924 23:15:07.135128 2711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7: Bootstrap starting.
I20260924 23:15:07.135890 2711 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7: Neither blocks nor log segments found. Creating new log.
I20260924 23:15:07.136204 2711 log.cc:824] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7: Log is configured to *not* fsync() on all Append() calls
I20260924 23:15:07.136963 2711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7: No bootstrap required, opened a new log
I20260924 23:15:07.139312 2711 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4671a395739f4c9cb7316315ce5645d7" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42845 } }
I20260924 23:15:07.139487 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:07.133393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47377 (local address 127.12.212.62:42845)
0924 23:15:07.133602 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:07.133613 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:07.133650 (+ 37us) server_negotiation.cc:407] Connection header received
0924 23:15:07.134222 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:07.134232 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:07.134306 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:07.134474 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:07.134737 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:07.136070 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:07.136996 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:07.137605 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:07.137729 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:07.138569 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:07.138574 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:07.138574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:15:07.138829 (+ 255us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:07.138876 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:15:07.138894 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:07.138981 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:07.139142 (+ 161us) server_negotiation.cc:299] Negotiation successful
0924 23:15:07.139226 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260924 23:15:07.139533 2711 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:15:07.139568 2711 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4671a395739f4c9cb7316315ce5645d7, State: Initialized, Role: FOLLOWER
I20260924 23:15:07.139678 2711 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4671a395739f4c9cb7316315ce5645d7" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42845 } }
I20260924 23:15:07.139743 2711 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:15:07.139773 2711 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:15:07.139822 2711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:15:07.140556 2711 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4671a395739f4c9cb7316315ce5645d7" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42845 } }
I20260924 23:15:07.140677 2711 leader_election.cc:302] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4671a395739f4c9cb7316315ce5645d7; no voters:
I20260924 23:15:07.140880 2711 leader_election.cc:288] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:15:07.140874 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:15:07.140944 2716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:15:07.141103 2716 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7 [term 1 LEADER]: Becoming Leader. State: Replica: 4671a395739f4c9cb7316315ce5645d7, State: Running, Role: LEADER
I20260924 23:15:07.141140 2711 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:15:07.141217 2716 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4671a395739f4c9cb7316315ce5645d7" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42845 } }
I20260924 23:15:07.141577 2716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4671a395739f4c9cb7316315ce5645d7. Latest consensus state: current_term: 1 leader_uuid: "4671a395739f4c9cb7316315ce5645d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4671a395739f4c9cb7316315ce5645d7" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42845 } } }
I20260924 23:15:07.141598 2718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4671a395739f4c9cb7316315ce5645d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4671a395739f4c9cb7316315ce5645d7" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 42845 } } }
I20260924 23:15:07.141710 2718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7 [sys.catalog]: This master's current role is: LEADER
I20260924 23:15:07.141858 2716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7 [sys.catalog]: This master's current role is: LEADER
I20260924 23:15:07.142033 2725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:15:07.142961 2725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:15:07.144873 2725 catalog_manager.cc:1347] Generated new cluster ID: 2dbfd4abe3cc4eebab4cfc456f8529d3
I20260924 23:15:07.144940 2725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:15:07.146090 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:07.142898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40391 (local address 127.12.212.62:42845)
0924 23:15:07.142942 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:07.142944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:07.143089 (+ 145us) server_negotiation.cc:407] Connection header received
0924 23:15:07.143184 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:07.143185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:07.143197 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:07.143225 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:07.143450 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:07.144051 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:07.144937 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:07.145149 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:07.145204 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:07.145853 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:07.145854 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:07.145855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:15:07.145860 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:07.145876 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:15:07.145880 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:07.145924 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:07.145987 (+ 63us) server_negotiation.cc:299] Negotiation successful
0924 23:15:07.146013 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 23:15:07.177591 2725 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:15:07.178192 2725 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:15:07.185038 2725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4671a395739f4c9cb7316315ce5645d7: Generated new TSK 0
I20260924 23:15:07.185262 2725 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:15:07.195091 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 2643
2026-09-24T23:15:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1294 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:15:07 dist-test-slave-m0z6 krb5kdc[2742](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:15:07 dist-test-slave-m0z6 krb5kdc[2742](info): set up 2 sockets
Sep 24 23:15:07 dist-test-slave-m0z6 krb5kdc[2742](info): commencing operation
krb5kdc: starting...
W20260924 23:15:08.162609 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.950s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:15:08 dist-test-slave-m0z6 krb5kdc[2742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:15:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:15:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:15:08.283710 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:33655
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:33655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:15:08.365227 2755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:15:08.365470 2755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:15:08.365496 2755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:15:08.366823 2755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:15:08.366880 2755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:15:08.366894 2755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:15:08.366907 2755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:15:08.366918 2755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:15:08.368600 2755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:33655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:33655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:15:08.368929 2755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:15:08.369170 2755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:15:08.374365 2763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:15:08.374509 2755 server_base.cc:1034] running on GCE node
W20260924 23:15:08.374598 2760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:08.374384 2761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:15:08.374893 2755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:15:08.375213 2755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:15:08.376389 2755 hybrid_clock.cc:648] HybridClock initialized: now 1790291708376373 us; error 40 us; skew 500 ppm
Sep 24 23:15:08 dist-test-slave-m0z6 krb5kdc[2742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291708, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:15:08.379088 2755 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:15:08.379582 2755 webserver.cc:460] Webserver started at http://127.12.212.62:37875/ using document root <none> and password file <none>
I20260924 23:15:08.379937 2755 fs_manager.cc:359] Metadata directory not provided
I20260924 23:15:08.380016 2755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:15:08.380148 2755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:15:08.381335 2755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "a84101152eb249f9ba0d8c201374a20f"
format_stamp: "Formatted at 2026-09-24 23:15:08 on dist-test-slave-m0z6"
server_key: "b04a42c18dcd62d99f96a0ca83b407c4"
server_key_iv: "cc86683cbb94424ee961516188c7d227"
server_key_version: "encryptionkey@0"
I20260924 23:15:08.381687 2755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "a84101152eb249f9ba0d8c201374a20f"
format_stamp: "Formatted at 2026-09-24 23:15:08 on dist-test-slave-m0z6"
server_key: "b04a42c18dcd62d99f96a0ca83b407c4"
server_key_iv: "cc86683cbb94424ee961516188c7d227"
server_key_version: "encryptionkey@0"
I20260924 23:15:08.383304 2755 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:15:08.384159 2770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:15:08.384367 2755 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:15:08.384481 2755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "a84101152eb249f9ba0d8c201374a20f"
format_stamp: "Formatted at 2026-09-24 23:15:08 on dist-test-slave-m0z6"
server_key: "b04a42c18dcd62d99f96a0ca83b407c4"
server_key_iv: "cc86683cbb94424ee961516188c7d227"
server_key_version: "encryptionkey@0"
I20260924 23:15:08.384562 2755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:15:08.388024 2755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:15:08.390154 2755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:15:08.390277 2755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:15:08.394371 2755 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:33655
I20260924 23:15:08.394485 2822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:33655 every 8 connection(s)
I20260924 23:15:08.394868 2755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:15:08.395919 2823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:15:08.398300 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 2755
I20260924 23:15:08.398404 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:15:08.398391 2823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f: Bootstrap starting.
I20260924 23:15:08.398571 13136 external_mini_cluster.cc:1428] Setting key 9a6068eba7e748f3b5bc8ae0a99e2dee
I20260924 23:15:08.399087 2823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f: Neither blocks nor log segments found. Creating new log.
I20260924 23:15:08.399428 2823 log.cc:824] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f: Log is configured to *not* fsync() on all Append() calls
I20260924 23:15:08.400307 2823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f: No bootstrap required, opened a new log
I20260924 23:15:08.401160 2826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:08.399124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59465 (local address 127.12.212.62:33655)
0924 23:15:08.399303 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:08.399311 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:08.399338 (+ 27us) server_negotiation.cc:407] Connection header received
0924 23:15:08.399823 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:08.399831 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:08.399862 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:08.400009 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:08.400726 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:08.400730 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:08.400730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:15:08.400734 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:08.400775 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:15:08.400787 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:08.400866 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:08.400920 (+ 54us) server_negotiation.cc:299] Negotiation successful
0924 23:15:08.400965 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260924 23:15:08.402232 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:15:08.402729 2823 raft_consensus.cc:348] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84101152eb249f9ba0d8c201374a20f" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 33655 } }
I20260924 23:15:08.402895 2823 raft_consensus.cc:374] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 23:15:08.402930 2823 raft_consensus.cc:729] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a84101152eb249f9ba0d8c201374a20f, State: Initialized, Role: FOLLOWER
I20260924 23:15:08.403056 2823 consensus_queue.cc:260] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84101152eb249f9ba0d8c201374a20f" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 33655 } }
I20260924 23:15:08.403138 2823 raft_consensus.cc:388] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 23:15:08.403172 2823 raft_consensus.cc:482] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 23:15:08.403221 2823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:15:08.403990 2823 raft_consensus.cc:504] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84101152eb249f9ba0d8c201374a20f" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 33655 } }
I20260924 23:15:08.404094 2823 leader_election.cc:302] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a84101152eb249f9ba0d8c201374a20f; no voters:
I20260924 23:15:08.404260 2823 leader_election.cc:288] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 23:15:08.404297 2833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:15:08.404462 2833 raft_consensus.cc:686] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f [term 1 LEADER]: Becoming Leader. State: Replica: a84101152eb249f9ba0d8c201374a20f, State: Running, Role: LEADER
I20260924 23:15:08.404522 2823 sys_catalog.cc:564] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:15:08.404577 2833 consensus_queue.cc:237] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84101152eb249f9ba0d8c201374a20f" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 33655 } }
I20260924 23:15:08.404983 2826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:08.403764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57819 (local address 127.12.212.62:33655)
0924 23:15:08.403845 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:08.403846 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:08.403960 (+ 114us) server_negotiation.cc:407] Connection header received
0924 23:15:08.404018 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:08.404019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:08.404024 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:08.404077 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:08.404673 (+ 596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:08.404674 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:08.404675 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 23:15:08.404679 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:08.404692 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 23:15:08.404696 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:08.404741 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:08.404912 (+ 171us) server_negotiation.cc:299] Negotiation successful
0924 23:15:08.404924 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 23:15:08.405184 2835 sys_catalog.cc:455] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a84101152eb249f9ba0d8c201374a20f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84101152eb249f9ba0d8c201374a20f" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 33655 } } }
I20260924 23:15:08.405313 2835 sys_catalog.cc:458] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f [sys.catalog]: This master's current role is: LEADER
I20260924 23:15:08.405359 2834 sys_catalog.cc:455] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a84101152eb249f9ba0d8c201374a20f. Latest consensus state: current_term: 1 leader_uuid: "a84101152eb249f9ba0d8c201374a20f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84101152eb249f9ba0d8c201374a20f" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 33655 } } }
I20260924 23:15:08.405433 2834 sys_catalog.cc:458] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f [sys.catalog]: This master's current role is: LEADER
I20260924 23:15:08.405562 2842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:15:08.406040 2842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:15:08.407575 2842 catalog_manager.cc:1347] Generated new cluster ID: 77d987a16d07410b8afed66d30e5d5ee
I20260924 23:15:08.407629 2842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:15:08.415701 2842 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:15:08.415915 2842 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:15:08.419874 2842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a84101152eb249f9ba0d8c201374a20f: Generated new TSK 0
I20260924 23:15:08.420076 2842 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:15:08.429833 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 2755
2026-09-24T23:15:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1234 ms)
[----------] 18 tests from AuthTokenIssuingTest (22322 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:15:08 dist-test-slave-m0z6 krb5kdc[2854](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:15:08 dist-test-slave-m0z6 krb5kdc[2854](info): set up 2 sockets
Sep 24 23:15:08 dist-test-slave-m0z6 krb5kdc[2854](info): commencing operation
krb5kdc: starting...
W20260924 23:15:09.386623 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:15:09 dist-test-slave-m0z6 krb5kdc[2854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:15:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:15:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:15:09.510900 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:38271
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:43875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:38271,127.12.212.61:34615,127.12.212.60:34365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:15:09.593505 2867 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 23:15:09.593758 2867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:15:09.593776 2867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:15:09.593797 2867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:15:09.595203 2867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:15:09.595263 2867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:15:09.595275 2867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:15:09.595287 2867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:15:09.595299 2867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:15:09.596958 2867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:43875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:38271,127.12.212.61:34615,127.12.212.60:34365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:38271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:15:09.597201 2867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:15:09.597437 2867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:15:09.602394 2867 server_base.cc:1034] running on GCE node
W20260924 23:15:09.602358 2873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:09.602358 2872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:09.602341 2875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:15:09.602855 2867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:15:09.603178 2867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:15:09.604346 2867 hybrid_clock.cc:648] HybridClock initialized: now 1790291709604303 us; error 60 us; skew 500 ppm
Sep 24 23:15:09 dist-test-slave-m0z6 krb5kdc[2854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291709, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:15:09.607002 2867 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:15:09.607450 2867 webserver.cc:460] Webserver started at http://127.12.212.62:41721/ using document root <none> and password file <none>
I20260924 23:15:09.607671 2867 fs_manager.cc:359] Metadata directory not provided
I20260924 23:15:09.607721 2867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:15:09.607848 2867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:15:09.608927 2867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "811b1b632de8486cbd1eb0d0aabc0242"
format_stamp: "Formatted at 2026-09-24 23:15:09 on dist-test-slave-m0z6"
server_key: "ac887e3f5934504fbb6da184def56035"
server_key_iv: "f53817fd94ef50471bae687a50e5f6f6"
server_key_version: "encryptionkey@0"
I20260924 23:15:09.609261 2867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "811b1b632de8486cbd1eb0d0aabc0242"
format_stamp: "Formatted at 2026-09-24 23:15:09 on dist-test-slave-m0z6"
server_key: "ac887e3f5934504fbb6da184def56035"
server_key_iv: "f53817fd94ef50471bae687a50e5f6f6"
server_key_version: "encryptionkey@0"
I20260924 23:15:09.610580 2867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:15:09.611438 2882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:15:09.611718 2867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:15:09.611824 2867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "811b1b632de8486cbd1eb0d0aabc0242"
format_stamp: "Formatted at 2026-09-24 23:15:09 on dist-test-slave-m0z6"
server_key: "ac887e3f5934504fbb6da184def56035"
server_key_iv: "f53817fd94ef50471bae687a50e5f6f6"
server_key_version: "encryptionkey@0"
I20260924 23:15:09.611898 2867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:15:09.642409 2867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:15:09.644728 2867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:15:09.644872 2867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:15:09.648888 2867 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:38271
I20260924 23:15:09.648954 2934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:38271 every 8 connection(s)
I20260924 23:15:09.649305 2867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:15:09.650275 2935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:15:09.651897 2935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:15:09.656145 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 2867
I20260924 23:15:09.656260 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:15:09.656399 13136 external_mini_cluster.cc:1428] Setting key 86a25415731e7a6591478baef4df4a1f
Sep 24 23:15:09 dist-test-slave-m0z6 krb5kdc[2854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291709, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:15:09.662297 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.652747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55108 (local address 127.12.212.62:38271)
0924 23:15:09.652977 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:09.652986 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:09.653144 (+ 158us) server_negotiation.cc:407] Connection header received
0924 23:15:09.653814 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:09.653817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:09.653836 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:09.653921 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:09.654533 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.655442 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.656446 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.656912 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.657044 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:09.659283 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:09.659291 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:09.659291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:09.659299 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:09.660760 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:09.661135 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:09.661138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:09.661139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:09.661240 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:09.661449 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:09.661450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:09.661450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:09.661670 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:09.661741 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:09.661925 (+ 184us) server_negotiation.cc:299] Negotiation successful
0924 23:15:09.662037 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":119,"threads_started":1}
I20260924 23:15:09.662298 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.652652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:38271 (local address 127.0.0.1:55108)
0924 23:15:09.652923 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.652975 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 23:15:09.653613 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:15:09.653979 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:15:09.653982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:15:09.654255 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:15:09.654424 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:09.654429 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.655554 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.655555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:09.656356 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:09.656359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.657026 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.657028 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:09.657085 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:09.657600 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:15:09.657608 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:15:09.659194 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:15:09.660845 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:09.660852 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:09.660855 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:09.661057 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:09.661304 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:09.661305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:09.661306 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:09.661386 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:09.661734 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:15:09.661739 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:15:09.661803 (+ 64us) client_negotiation.cc:769] Sending connection context
0924 23:15:09.661883 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 23:15:09.662114 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":75,"thread_start_us":80,"threads_started":1}
I20260924 23:15:09.663101 2935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.212.61@KRBTEST.COM; defaulting to no policy
I20260924 23:15:09.663497 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.663383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:34615 (local address 127.0.0.1:59162)
0924 23:15:09.663425 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.663473 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260924 23:15:09.663645 2883 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34615: connect: Connection refused (error 111)
I20260924 23:15:09.664074 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.663987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:34615 (local address 127.0.0.1:59168)
0924 23:15:09.664033 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.664057 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260924 23:15:09.664161 2935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.61:34615: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34615: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.61.keytab.
Entry for principal kudu/127.12.212.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.61.keytab.
I20260924 23:15:09.681293 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.61:34615
--webserver_interface=127.12.212.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.61.keytab
--principal=kudu/127.12.212.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:43875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:38271,127.12.212.61:34615,127.12.212.60:34365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 23:15:09.713418 2935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } attempt: 1
I20260924 23:15:09.713943 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.713802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:34615 (local address 127.0.0.1:59180)
0924 23:15:09.713866 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.713903 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260924 23:15:09.714356 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.714259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:34615 (local address 127.0.0.1:59184)
0924 23:15:09.714310 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.714336 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260924 23:15:09.714454 2935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.61:34615: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34615: connect: Connection refused (error 111)
W20260924 23:15:09.763398 2942 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 23:15:09.763664 2942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:15:09.763691 2942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:15:09.763722 2942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:15:09.765040 2942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:15:09.765100 2942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:15:09.765118 2942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:15:09.765134 2942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:15:09.765149 2942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:15:09.766949 2942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:43875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:38271,127.12.212.61:34615,127.12.212.60:34365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.61.keytab
--principal=kudu/127.12.212.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.61:34615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.212.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:15:09.767243 2942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:15:09.767479 2942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:15:09.772442 2950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:09.772493 2947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:09.772462 2948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:15:09.772675 2942 server_base.cc:1034] running on GCE node
I20260924 23:15:09.772921 2942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:15:09.773242 2942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:15:09.774417 2942 hybrid_clock.cc:648] HybridClock initialized: now 1790291709774402 us; error 32 us; skew 500 ppm
Sep 24 23:15:09 dist-test-slave-m0z6 krb5kdc[2854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291709, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:15:09.777136 2942 init.cc:377] Logged in from keytab as kudu/127.12.212.61@KRBTEST.COM (short username kudu)
I20260924 23:15:09.777642 2942 webserver.cc:460] Webserver started at http://127.12.212.61:36473/ using document root <none> and password file <none>
I20260924 23:15:09.777865 2942 fs_manager.cc:359] Metadata directory not provided
I20260924 23:15:09.777925 2942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:15:09.778053 2942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:15:09.779179 2942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-1/data/instance:
uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf"
format_stamp: "Formatted at 2026-09-24 23:15:09 on dist-test-slave-m0z6"
server_key: "2a0a227a993ecab329036f7564e3e356"
server_key_iv: "a55f10e028d2f38090a55c712cba531f"
server_key_version: "encryptionkey@0"
I20260924 23:15:09.779516 2942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-1/wal/instance:
uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf"
format_stamp: "Formatted at 2026-09-24 23:15:09 on dist-test-slave-m0z6"
server_key: "2a0a227a993ecab329036f7564e3e356"
server_key_iv: "a55f10e028d2f38090a55c712cba531f"
server_key_version: "encryptionkey@0"
I20260924 23:15:09.780853 2942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 23:15:09.781646 2957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:15:09.781821 2942 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:15:09.781906 2942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-1/wal
uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf"
format_stamp: "Formatted at 2026-09-24 23:15:09 on dist-test-slave-m0z6"
server_key: "2a0a227a993ecab329036f7564e3e356"
server_key_iv: "a55f10e028d2f38090a55c712cba531f"
server_key_version: "encryptionkey@0"
I20260924 23:15:09.781971 2942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:15:09.782588 2935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } attempt: 2
I20260924 23:15:09.783013 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.782868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:34615 (local address 127.0.0.1:59196)
0924 23:15:09.782930 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.782977 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260924 23:15:09.783326 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.783242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:34615 (local address 127.0.0.1:59198)
0924 23:15:09.783284 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.783307 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260924 23:15:09.783413 2935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.61:34615: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34615: connect: Connection refused (error 111)
I20260924 23:15:09.809013 2942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:15:09.811337 2942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:15:09.811456 2942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:15:09.815475 2942 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.61:34615
I20260924 23:15:09.815516 3009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.61:34615 every 8 connection(s)
I20260924 23:15:09.815919 2942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-1/data/info.pb
I20260924 23:15:09.816363 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 2942
I20260924 23:15:09.816447 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-1/wal/instance
I20260924 23:15:09.816617 13136 external_mini_cluster.cc:1428] Setting key 00200850b314e0990329455f4ec9c97c
I20260924 23:15:09.817008 3010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:15:09.818584 3010 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.212.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.60@KRBTEST.COM" created.
Sep 24 23:15:09 dist-test-slave-m0z6 krb5kdc[2854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291709, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.61@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:15:09.831014 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.819447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55110 (local address 127.12.212.62:38271)
0924 23:15:09.819503 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:09.819504 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:09.819727 (+ 223us) server_negotiation.cc:407] Connection header received
0924 23:15:09.820250 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:09.820251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:09.820269 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:09.820323 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:09.821046 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.821549 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.822607 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.822881 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.822945 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:09.827474 (+ 4529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:09.827477 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:09.827477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:09.827486 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:09.829059 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:09.829629 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:09.829630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:09.829631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:09.829695 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:09.830077 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:09.830077 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:09.830078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:09.830202 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:09.830256 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:09.830723 (+ 467us) server_negotiation.cc:299] Negotiation successful
0924 23:15:09.830817 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 23:15:09.831122 3013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.819407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:38271 (local address 127.0.0.1:55110)
0924 23:15:09.819598 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.819638 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 23:15:09.820140 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:15:09.820404 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:15:09.820409 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:15:09.820680 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:15:09.820921 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:09.820926 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.821659 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.821661 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:09.822506 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:09.822508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.822983 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.822984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:09.823120 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:09.823645 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:15:09.823654 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:15:09.827312 (+ 3658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:15:09.829165 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:09.829179 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:09.829182 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:09.829527 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:09.829767 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:09.829768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:09.829769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:09.830004 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:09.830265 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:15:09.830273 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:15:09.830468 (+ 195us) client_negotiation.cc:769] Sending connection context
0924 23:15:09.830615 (+ 147us) client_negotiation.cc:240] Negotiation successful
0924 23:15:09.830853 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":93,"threads_started":1}
I20260924 23:15:09.831733 3010 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 23:15:09 dist-test-slave-m0z6 krb5kdc[2854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291709, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.61@KRBTEST.COM for kudu/127.12.212.61@KRBTEST.COM
I20260924 23:15:09.841398 3013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.832015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:34615 (local address 127.0.0.1:59212)
0924 23:15:09.832063 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.832074 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:15:09.832106 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:15:09.832421 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:15:09.832422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:15:09.832572 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:15:09.832630 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:09.832632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.833523 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.833524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:09.834031 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:09.834033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.834525 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.834526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:09.834589 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:09.835052 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:15:09.835058 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:15:09.838376 (+ 3318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:15:09.840258 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:09.840259 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:09.840259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:09.840437 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:09.840669 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:09.840669 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:09.840670 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:09.840720 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:09.841012 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:15:09.841012 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:15:09.841052 (+ 40us) client_negotiation.cc:769] Sending connection context
0924 23:15:09.841110 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 23:15:09.841308 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":65}
I20260924 23:15:09.841404 3015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.832105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59212 (local address 127.12.212.61:34615)
0924 23:15:09.832289 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:09.832293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:09.832314 (+ 21us) server_negotiation.cc:407] Connection header received
0924 23:15:09.832332 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:09.832333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:09.832344 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:09.832438 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:09.832699 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.833429 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.834107 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.834449 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.834513 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:09.838491 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:09.838506 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:09.838507 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:09.838514 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:09.840172 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:09.840521 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:09.840525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:09.840526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:09.840581 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:09.840775 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:09.840776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:09.840776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:09.840951 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:09.841025 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:09.841152 (+ 127us) server_negotiation.cc:299] Negotiation successful
0924 23:15:09.841243 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":105,"threads_started":1}
I20260924 23:15:09.842252 3010 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:15:09.842677 3013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.842549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34365 (local address 127.0.0.1:52064)
0924 23:15:09.842600 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.842650 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260924 23:15:09.842852 2958 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34365: connect: Connection refused (error 111)
I20260924 23:15:09.843317 3013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.843224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34365 (local address 127.0.0.1:52066)
0924 23:15:09.843265 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.843291 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260924 23:15:09.843456 3010 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.60:34365: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34365: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.60.keytab.
Entry for principal kudu/127.12.212.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.60.keytab.
I20260924 23:15:09.847584 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.60:34365
--webserver_interface=127.12.212.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.60.keytab
--principal=kudu/127.12.212.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:43875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:38271,127.12.212.61:34615,127.12.212.60:34365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 23:15:09.874598 2935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } attempt: 3
Sep 24 23:15:09 dist-test-slave-m0z6 krb5kdc[2854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291709, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for kudu/127.12.212.61@KRBTEST.COM
I20260924 23:15:09.882035 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.875049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:34615 (local address 127.0.0.1:59220)
0924 23:15:09.875104 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.875117 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 23:15:09.875163 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:15:09.875312 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:15:09.875313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:15:09.875478 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:15:09.875544 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:09.875547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.876218 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.876220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:09.876781 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:09.876785 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.877185 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.877186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:09.877259 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:09.877699 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:15:09.877704 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:15:09.879022 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:15:09.880987 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:09.880989 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:09.880989 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:09.881179 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:09.881396 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:09.881397 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:09.881397 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:09.881460 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:09.881722 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:15:09.881722 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:15:09.881766 (+ 44us) client_negotiation.cc:769] Sending connection context
0924 23:15:09.881813 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 23:15:09.881898 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 23:15:09.882033 3015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.875127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59220 (local address 127.12.212.61:34615)
0924 23:15:09.875195 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:09.875197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:09.875207 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:15:09.875234 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:09.875235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:09.875247 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:09.875290 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:09.875626 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.876138 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.876887 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.877111 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.877169 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:09.879132 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:09.879134 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:09.879134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:09.879144 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:09.880826 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:09.881259 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:09.881260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:09.881261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:09.881335 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:09.881517 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:09.881518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:09.881518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:09.881662 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:09.881729 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:09.881854 (+ 125us) server_negotiation.cc:299] Negotiation successful
0924 23:15:09.881939 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 23:15:09.882448 2935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:15:09.882808 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.882688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34365 (local address 127.0.0.1:52070)
0924 23:15:09.882749 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.882783 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260924 23:15:09.883152 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.883066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34365 (local address 127.0.0.1:52082)
0924 23:15:09.883110 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.883134 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260924 23:15:09.883245 2935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.60:34365: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34365: connect: Connection refused (error 111)
I20260924 23:15:09.892702 3010 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } attempt: 1
I20260924 23:15:09.893270 3013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.893121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34365 (local address 127.0.0.1:52084)
0924 23:15:09.893181 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.893231 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260924 23:15:09.893687 3013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.893599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34365 (local address 127.0.0.1:52096)
0924 23:15:09.893639 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.893666 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260924 23:15:09.893848 3010 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.60:34365: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34365: connect: Connection refused (error 111)
I20260924 23:15:09.914386 2935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } attempt: 1
I20260924 23:15:09.914954 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.914801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34365 (local address 127.0.0.1:52108)
0924 23:15:09.914861 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.914915 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 23:15:09.915423 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.915299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34365 (local address 127.0.0.1:52110)
0924 23:15:09.915366 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.915398 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260924 23:15:09.915545 2935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.60:34365: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34365: connect: Connection refused (error 111)
W20260924 23:15:09.933740 3017 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 23:15:09.933985 3017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:15:09.934003 3017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:15:09.934024 3017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:15:09.935320 3017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:15:09.935379 3017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:15:09.935395 3017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:15:09.935405 3017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:15:09.935416 3017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:15:09.937037 3017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:43875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:38271,127.12.212.61:34615,127.12.212.60:34365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.60.keytab
--principal=kudu/127.12.212.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.60:34365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.212.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:15:09.937278 3017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:15:09.937520 3017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:15:09.942514 3022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:09.942514 3023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:09.942528 3025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:15:09.942795 3017 server_base.cc:1034] running on GCE node
I20260924 23:15:09.943112 3017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:15:09.943429 3017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:15:09.944619 3017 hybrid_clock.cc:648] HybridClock initialized: now 1790291709944605 us; error 30 us; skew 500 ppm
Sep 24 23:15:09 dist-test-slave-m0z6 krb5kdc[2854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291709, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:15:09.947546 3017 init.cc:377] Logged in from keytab as kudu/127.12.212.60@KRBTEST.COM (short username kudu)
I20260924 23:15:09.948158 3017 webserver.cc:460] Webserver started at http://127.12.212.60:40427/ using document root <none> and password file <none>
I20260924 23:15:09.948407 3017 fs_manager.cc:359] Metadata directory not provided
I20260924 23:15:09.948457 3017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:15:09.948576 3017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:15:09.949754 3017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-2/data/instance:
uuid: "fe936726ab3744f9b0ae8d71852aa4a8"
format_stamp: "Formatted at 2026-09-24 23:15:09 on dist-test-slave-m0z6"
server_key: "87faa81bf78e09eef810fd9d3a0087f1"
server_key_iv: "deb89f4cbddae67b101b914db7b8fa30"
server_key_version: "encryptionkey@0"
I20260924 23:15:09.950095 3017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-2/wal/instance:
uuid: "fe936726ab3744f9b0ae8d71852aa4a8"
format_stamp: "Formatted at 2026-09-24 23:15:09 on dist-test-slave-m0z6"
server_key: "87faa81bf78e09eef810fd9d3a0087f1"
server_key_iv: "deb89f4cbddae67b101b914db7b8fa30"
server_key_version: "encryptionkey@0"
I20260924 23:15:09.951512 3017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:15:09.952371 3032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:15:09.952582 3017 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:15:09.952674 3017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-2/wal
uuid: "fe936726ab3744f9b0ae8d71852aa4a8"
format_stamp: "Formatted at 2026-09-24 23:15:09 on dist-test-slave-m0z6"
server_key: "87faa81bf78e09eef810fd9d3a0087f1"
server_key_iv: "deb89f4cbddae67b101b914db7b8fa30"
server_key_version: "encryptionkey@0"
I20260924 23:15:09.952752 3017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:15:09.962045 3010 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } attempt: 2
I20260924 23:15:09.962548 3013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.962409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34365 (local address 127.0.0.1:52118)
0924 23:15:09.962475 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.962518 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260924 23:15:09.962977 3013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.962828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34365 (local address 127.0.0.1:52128)
0924 23:15:09.962928 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.962955 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260924 23:15:09.963083 3010 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.60:34365: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34365: connect: Connection refused (error 111)
I20260924 23:15:09.965104 3017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:15:09.967374 3017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:15:09.967549 3017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:15:09.971587 3017 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.60:34365
I20260924 23:15:09.971617 3084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.60:34365 every 8 connection(s)
I20260924 23:15:09.971958 3017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-2/data/info.pb
I20260924 23:15:09.972999 3085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:15:09.972988 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 3017
I20260924 23:15:09.973083 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790291444018253-13136-0/minicluster-data/master-2/wal/instance
I20260924 23:15:09.973260 13136 external_mini_cluster.cc:1428] Setting key add08231dda423c4d23ad7b7102aaddb
I20260924 23:15:09.975207 3085 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 23:15:09 dist-test-slave-m0z6 krb5kdc[2854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.60@KRBTEST.COM
Sep 24 23:15:09 dist-test-slave-m0z6 krb5kdc[2854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291709, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.60@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:15:09.984992 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.975866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55122 (local address 127.12.212.62:38271)
0924 23:15:09.975918 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:09.975920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:09.976071 (+ 151us) server_negotiation.cc:407] Connection header received
0924 23:15:09.976106 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:09.976107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:09.976122 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:09.976164 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:09.976719 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.977444 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.978582 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.978881 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.978939 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:09.981895 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:09.981897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:09.981897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:09.981907 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:09.983512 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:09.983950 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:09.983951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:09.983951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:09.984012 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:09.984303 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:09.984304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:09.984305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:09.984503 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:09.984573 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:09.984801 (+ 228us) server_negotiation.cc:299] Negotiation successful
0924 23:15:09.984891 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 23:15:09.985307 3089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.975768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:38271 (local address 127.0.0.1:55122)
0924 23:15:09.975905 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.975952 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 23:15:09.976026 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:15:09.976203 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:15:09.976206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:15:09.976564 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:15:09.976627 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:09.976629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.977559 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.977560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:09.978483 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:09.978486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.978996 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.978997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:09.979076 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:09.979548 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:15:09.979557 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:15:09.981693 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:15:09.983595 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:09.983603 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:09.983606 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:09.983861 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:09.984091 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:09.984092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:09.984092 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:09.984214 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:09.984580 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:15:09.984586 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:15:09.984656 (+ 70us) client_negotiation.cc:769] Sending connection context
0924 23:15:09.984771 (+ 115us) client_negotiation.cc:240] Negotiation successful
0924 23:15:09.985096 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":26,"thread_start_us":72,"threads_started":1}
I20260924 23:15:09.985571 3088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.973798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52134 (local address 127.12.212.60:34365)
0924 23:15:09.973988 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:09.973996 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:09.974022 (+ 26us) server_negotiation.cc:407] Connection header received
0924 23:15:09.974476 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:09.974484 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:09.974535 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:09.974667 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:09.975328 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.976605 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.977683 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.978373 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.978490 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:09.981293 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:09.981303 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:09.981304 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:09.981312 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:09.983424 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:09.983924 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:09.983929 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:09.983929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:09.984067 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:09.984303 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:09.984304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:09.984305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:09.984623 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:09.984735 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:09.984873 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 23:15:09.985487 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":421,"server-negotiator.queue_time_us":142,"thread_start_us":85,"threads_started":1}
I20260924 23:15:09.985958 3085 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 23:15:09 dist-test-slave-m0z6 krb5kdc[2854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291709, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.60@KRBTEST.COM for kudu/127.12.212.61@KRBTEST.COM
Sep 24 23:15:09 dist-test-slave-m0z6 krb5kdc[2854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:15:09.990724 2935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } attempt: 2
I20260924 23:15:09.993516 3015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.986293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59230 (local address 127.12.212.61:34615)
0924 23:15:09.986421 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:09.986423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:09.986432 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:15:09.986451 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:09.986452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:09.986460 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:09.986501 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:09.986799 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.987337 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.988142 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.988352 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.988414 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:09.990462 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:09.990463 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:09.990464 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:09.990471 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:09.992244 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:09.992702 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:09.992703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:09.992704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:09.992767 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:09.992968 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:09.992970 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:09.992970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:09.993119 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:09.993192 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:09.993346 (+ 154us) server_negotiation.cc:299] Negotiation successful
0924 23:15:09.993421 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260924 23:15:09.993536 3089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.986236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:34615 (local address 127.0.0.1:59230)
0924 23:15:09.986271 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.986282 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:15:09.986318 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:15:09.986520 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:15:09.986521 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:15:09.986671 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:15:09.986739 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:09.986741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.987415 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.987416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:09.988040 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:09.988043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.988426 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.988427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:09.988487 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:09.988910 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:15:09.988915 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:15:09.990361 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:15:09.992362 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:09.992363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:09.992363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:09.992590 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:09.992837 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:09.992838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:09.992838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:09.992897 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:09.993194 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:15:09.993194 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:15:09.993241 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 23:15:09.993306 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 23:15:09.993433 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260924 23:15:09.994390 3085 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:15:09.994583 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.986357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55134 (local address 127.12.212.62:38271)
0924 23:15:09.986390 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:09.986392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:09.986611 (+ 219us) server_negotiation.cc:407] Connection header received
0924 23:15:09.986663 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:09.986664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:09.986672 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:09.986765 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:09.987221 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.987737 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.988576 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.988773 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.988846 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:09.991220 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:09.991221 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:09.991221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:09.991228 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:09.993459 (+ 2231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:09.993853 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:09.993854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:09.993854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:09.993907 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:09.994105 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:09.994106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:09.994106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:09.994225 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:09.994393 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:09.994427 (+ 34us) server_negotiation.cc:299] Negotiation successful
0924 23:15:09.994497 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Sep 24 23:15:09 dist-test-slave-m0z6 krb5kdc[2854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291709, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for kudu/127.12.212.60@KRBTEST.COM
Sep 24 23:15:09 dist-test-slave-m0z6 krb5kdc[2854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291709, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.60@KRBTEST.COM for kudu/127.12.212.60@KRBTEST.COM
I20260924 23:15:09.998871 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.990991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34365 (local address 127.0.0.1:52144)
0924 23:15:09.991053 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.991064 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:15:09.991109 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:15:09.991210 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:15:09.991210 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:15:09.991407 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:15:09.991454 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:09.991457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.992391 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.992392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:09.993171 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:09.993174 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.993641 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.993642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:09.993697 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:09.994040 (+ 343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:15:09.994044 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:15:09.995488 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:15:09.997790 (+ 2302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:09.997791 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:09.997792 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:09.998009 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:09.998195 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:09.998196 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:09.998196 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:09.998254 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:09.998523 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:15:09.998524 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:15:09.998574 (+ 50us) client_negotiation.cc:769] Sending connection context
0924 23:15:09.998647 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 23:15:09.998773 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260924 23:15:09.998890 3088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.991049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52144 (local address 127.12.212.60:34365)
0924 23:15:09.991085 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:09.991086 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:09.991093 (+ 7us) server_negotiation.cc:407] Connection header received
0924 23:15:09.991154 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:09.991155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:09.991164 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:09.991207 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:09.991543 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.992308 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.993251 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.993547 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.993638 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:09.995565 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:09.995566 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:09.995567 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:09.995575 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:09.997686 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:09.998077 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:09.998078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:09.998079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:09.998135 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:09.998314 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:09.998316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:09.998316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:09.998452 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:09.998518 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:09.998720 (+ 202us) server_negotiation.cc:299] Negotiation successful
0924 23:15:09.998801 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 24 23:15:09 dist-test-slave-m0z6 krb5kdc[2854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.61@KRBTEST.COM
I20260924 23:15:10.000955 2935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242: Bootstrap starting.
I20260924 23:15:10.001780 2935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242: Neither blocks nor log segments found. Creating new log.
I20260924 23:15:10.002107 3089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.994849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34365 (local address 127.0.0.1:52150)
0924 23:15:09.994920 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:09.994931 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:15:09.994971 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:15:09.995216 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:15:09.995217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:15:09.995357 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:15:09.995398 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:09.995399 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.996589 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.996590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:09.997082 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:09.997084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.997435 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.997436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:09.997482 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:09.997801 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:15:09.997805 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:15:09.999205 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:15:10.000855 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:10.000856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:10.000856 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:10.001025 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:10.001203 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:10.001203 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:10.001204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:10.001253 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:10.001770 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:15:10.001771 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:15:10.001818 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 23:15:10.001887 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 23:15:10.002004 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260924 23:15:10.002121 2935 log.cc:824] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242: Log is configured to *not* fsync() on all Append() calls
I20260924 23:15:10.002858 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.994934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52150 (local address 127.12.212.60:34365)
0924 23:15:09.995104 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:09.995106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:09.995111 (+ 5us) server_negotiation.cc:407] Connection header received
0924 23:15:09.995126 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:09.995127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:09.995137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:09.995461 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:09.995498 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.996493 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.997161 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.997369 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.997425 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:09.999296 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:09.999297 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:09.999298 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:09.999304 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:10.000767 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:10.001093 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:10.001094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:10.001094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:10.001146 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:10.001311 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:10.001311 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:10.001312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:10.001672 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:10.001761 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:10.001929 (+ 168us) server_negotiation.cc:299] Negotiation successful
0924 23:15:10.002095 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":152,"thread_start_us":119,"threads_started":1}
I20260924 23:15:10.003126 2935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242: No bootstrap required, opened a new log
I20260924 23:15:10.004067 3015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:09.995648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59240 (local address 127.12.212.61:34615)
0924 23:15:09.995688 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:09.995690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:09.995855 (+ 165us) server_negotiation.cc:407] Connection header received
0924 23:15:09.995885 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:09.995886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:09.995897 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:09.995942 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:09.996546 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.997005 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.997943 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:09.998233 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:09.998294 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:10.000921 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:10.000922 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:10.000923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:10.000930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:10.002802 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:10.003248 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:10.003250 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:10.003251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:10.003316 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:10.003534 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:10.003534 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:10.003535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:10.003671 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:10.003855 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:10.003894 (+ 39us) server_negotiation.cc:299] Negotiation successful
0924 23:15:10.003977 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 23:15:10.004288 3085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8: Bootstrap starting.
I20260924 23:15:10.005023 3085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8: Neither blocks nor log segments found. Creating new log.
I20260924 23:15:10.005352 3085 log.cc:824] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8: Log is configured to *not* fsync() on all Append() calls
I20260924 23:15:10.005343 2935 raft_consensus.cc:348] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "811b1b632de8486cbd1eb0d0aabc0242" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } } peers { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } } peers { permanent_uuid: "fe936726ab3744f9b0ae8d71852aa4a8" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } }
I20260924 23:15:10.005501 2935 raft_consensus.cc:729] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 811b1b632de8486cbd1eb0d0aabc0242, State: Initialized, Role: FOLLOWER
I20260924 23:15:10.005604 2935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "811b1b632de8486cbd1eb0d0aabc0242" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } } peers { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } } peers { permanent_uuid: "fe936726ab3744f9b0ae8d71852aa4a8" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } }
I20260924 23:15:10.005736 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:15:10.005973 2935 sys_catalog.cc:564] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:15:10.006312 3085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8: No bootstrap required, opened a new log
I20260924 23:15:10.006667 3093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "811b1b632de8486cbd1eb0d0aabc0242" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } } peers { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } } peers { permanent_uuid: "fe936726ab3744f9b0ae8d71852aa4a8" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } } }
I20260924 23:15:10.006824 2909 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "811b1b632de8486cbd1eb0d0aabc0242"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55134
I20260924 23:15:10.006875 3093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:15:10.006963 2909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 23:15:10.007004 2909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:15:10.007786 2909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "811b1b632de8486cbd1eb0d0aabc0242" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } } peers { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } } peers { permanent_uuid: "fe936726ab3744f9b0ae8d71852aa4a8" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } }
I20260924 23:15:10.008073 2909 leader_election.cc:288] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [CANDIDATE]: Term 1 election: Requested vote from peers f42d5ebbc6f74ceab92b8d68acdfc7cf (127.12.212.61:34615), fe936726ab3744f9b0ae8d71852aa4a8 (127.12.212.60:34365)
I20260924 23:15:10.008414 3085 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "811b1b632de8486cbd1eb0d0aabc0242" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } } peers { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } } peers { permanent_uuid: "fe936726ab3744f9b0ae8d71852aa4a8" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } }
I20260924 23:15:10.008630 3085 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe936726ab3744f9b0ae8d71852aa4a8, State: Initialized, Role: FOLLOWER
I20260924 23:15:10.008764 3085 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "811b1b632de8486cbd1eb0d0aabc0242" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } } peers { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } } peers { permanent_uuid: "fe936726ab3744f9b0ae8d71852aa4a8" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } }
I20260924 23:15:10.009059 3107 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "811b1b632de8486cbd1eb0d0aabc0242" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } } peers { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } } peers { permanent_uuid: "fe936726ab3744f9b0ae8d71852aa4a8" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } } }
I20260924 23:15:10.009147 3085 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:15:10.009167 3107 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:15:10.009205 3059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "811b1b632de8486cbd1eb0d0aabc0242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fe936726ab3744f9b0ae8d71852aa4a8"
I20260924 23:15:10.009341 3059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:15:10.010291 3059 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 811b1b632de8486cbd1eb0d0aabc0242 in term 1.
W20260924 23:15:10.011152 3104 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:15:10.011229 3104 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:15:10.009707 2867 master_runner.cc:186] Error getting master registration for 127.12.212.61:34615: OK, instance_id { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" instance_seqno: 1790291709814307 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 23:15:10.012867 2867 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 23:15:10.011142 3120 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260924 23:15:10.013314 2883 leader_election.cc:302] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 811b1b632de8486cbd1eb0d0aabc0242, fe936726ab3744f9b0ae8d71852aa4a8; no voters:
I20260924 23:15:10.013545 3093 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:15:10.013751 3093 raft_consensus.cc:686] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [term 1 LEADER]: Becoming Leader. State: Replica: 811b1b632de8486cbd1eb0d0aabc0242, State: Running, Role: LEADER
I20260924 23:15:10.013859 3093 consensus_queue.cc:237] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "811b1b632de8486cbd1eb0d0aabc0242" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } } peers { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } } peers { permanent_uuid: "fe936726ab3744f9b0ae8d71852aa4a8" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } }
I20260924 23:15:10.014389 3093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 811b1b632de8486cbd1eb0d0aabc0242. Latest consensus state: current_term: 1 leader_uuid: "811b1b632de8486cbd1eb0d0aabc0242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "811b1b632de8486cbd1eb0d0aabc0242" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } } peers { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } } peers { permanent_uuid: "fe936726ab3744f9b0ae8d71852aa4a8" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } } }
I20260924 23:15:10.014499 3093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [sys.catalog]: This master's current role is: LEADER
W20260924 23:15:10.015633 3120 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:15:10.019345 3128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:15:10.016988 3017 master_runner.cc:186] Error getting master registration for 127.12.212.61:34615: OK, instance_id { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" instance_seqno: 1790291709814307 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 23:15:10.019544 3017 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 23:15:10.019567 3128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:15:10.027004 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:10.014754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55140 (local address 127.12.212.62:38271)
0924 23:15:10.014833 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:10.014835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:10.014884 (+ 49us) server_negotiation.cc:407] Connection header received
0924 23:15:10.015004 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:10.015005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:10.015016 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:10.015047 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:10.016897 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:10.017662 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:10.019877 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:10.020173 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:10.020217 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:10.023555 (+ 3338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:10.023557 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:10.023558 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:10.023568 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:10.025116 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:10.026108 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:10.026110 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:10.026110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:10.026173 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:10.026364 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:10.026365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:10.026365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:10.026498 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:10.026591 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:10.026822 (+ 231us) server_negotiation.cc:299] Negotiation successful
0924 23:15:10.026914 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260924 23:15:10.027818 3059 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8 [term 1 FOLLOWER]: Refusing update from remote peer 811b1b632de8486cbd1eb0d0aabc0242: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:15:10.028198 3123 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe936726ab3744f9b0ae8d71852aa4a8" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:15:10.029858 3107 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 811b1b632de8486cbd1eb0d0aabc0242. Latest consensus state: current_term: 1 leader_uuid: "811b1b632de8486cbd1eb0d0aabc0242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "811b1b632de8486cbd1eb0d0aabc0242" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } } peers { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } } peers { permanent_uuid: "fe936726ab3744f9b0ae8d71852aa4a8" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } } }
I20260924 23:15:10.029990 3107 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:15:10.031611 3129 mvcc.cc:205] Tried to move back new op lower bound from 7333034844270804992 to 7333034844217774080. Current Snapshot: MvccSnapshot[applied={T|T < 7333034844270804992}]
I20260924 23:15:10.031865 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:10.014383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52158 (local address 127.12.212.60:34365)
0924 23:15:10.014801 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:10.014803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:10.018949 (+ 4146us) server_negotiation.cc:407] Connection header received
0924 23:15:10.019054 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:10.019055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:10.019070 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:10.019101 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:10.019864 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:10.020541 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:10.021686 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:10.022022 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:10.022079 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:10.027404 (+ 5325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:10.027406 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:10.027406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:10.027417 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:10.029150 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:10.030903 (+ 1753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:10.030905 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:10.030905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:10.030978 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:10.031197 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:10.031197 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:10.031198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:10.031341 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:10.031456 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:10.031649 (+ 193us) server_negotiation.cc:299] Negotiation successful
0924 23:15:10.031764 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393}
I20260924 23:15:10.031971 3093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "811b1b632de8486cbd1eb0d0aabc0242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "811b1b632de8486cbd1eb0d0aabc0242" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } } peers { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } } peers { permanent_uuid: "fe936726ab3744f9b0ae8d71852aa4a8" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } } }
I20260924 23:15:10.032063 3093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [sys.catalog]: This master's current role is: LEADER
I20260924 23:15:10.032483 3107 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "811b1b632de8486cbd1eb0d0aabc0242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "811b1b632de8486cbd1eb0d0aabc0242" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } } peers { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } } peers { permanent_uuid: "fe936726ab3744f9b0ae8d71852aa4a8" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } } }
I20260924 23:15:10.032570 3107 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:15:10.033890 3130 mvcc.cc:205] Tried to move back new op lower bound from 7333034844270804992 to 7333034844217774080. Current Snapshot: MvccSnapshot[applied={T|T < 7333034844270804992}]
I20260924 23:15:10.043541 3128 catalog_manager.cc:1347] Generated new cluster ID: 6e715f3cff0446a7a36600fa4bfa5e6f
I20260924 23:15:10.043619 3128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:15:10.049494 3015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:10.014186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59254 (local address 127.12.212.61:34615)
0924 23:15:10.014249 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:10.014252 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:10.016412 (+ 2160us) server_negotiation.cc:407] Connection header received
0924 23:15:10.017816 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:10.017818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:10.017830 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:10.017897 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:10.020457 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:10.021183 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:10.027524 (+ 6341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:10.027843 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:10.027888 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:10.030247 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:10.030262 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:10.030262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:10.030271 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:10.042947 (+ 12676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:10.043483 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:10.043485 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:10.043485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:10.043571 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:10.048267 (+ 4696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:10.048269 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:10.048269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:10.048460 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:10.048620 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:10.049240 (+ 620us) server_negotiation.cc:299] Negotiation successful
0924 23:15:10.049385 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 23:15:10.054193 3010 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } attempt: 3
Sep 24 23:15:10 dist-test-slave-m0z6 krb5kdc[2854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291709, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.61@KRBTEST.COM for kudu/127.12.212.60@KRBTEST.COM
I20260924 23:15:10.065335 3013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:10.054542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34365 (local address 127.0.0.1:52174)
0924 23:15:10.054647 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:10.054659 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:15:10.054719 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:15:10.054981 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:15:10.054982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:15:10.055188 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:15:10.055244 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:10.055246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:10.057239 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:10.057240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:10.058074 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:10.058077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:10.058670 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:10.058671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:10.058755 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:10.059287 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:15:10.059301 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:15:10.060876 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:15:10.063112 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:10.063113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:10.063113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:10.063354 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:10.063598 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:10.063599 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:10.063600 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:10.063669 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:10.064945 (+ 1276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:15:10.064946 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:15:10.065025 (+ 79us) client_negotiation.cc:769] Sending connection context
0924 23:15:10.065089 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 23:15:10.065211 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260924 23:15:10.065745 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:10.054798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52174 (local address 127.12.212.60:34365)
0924 23:15:10.054841 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:10.054843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:10.054850 (+ 7us) server_negotiation.cc:407] Connection header received
0924 23:15:10.054877 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:10.054878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:10.054888 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:10.054932 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:10.055339 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:10.056126 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:10.058196 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:10.058541 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:10.058599 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:10.061003 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:10.061005 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:10.061005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:10.061030 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:10.063003 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:10.063446 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:10.063447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:10.063448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:10.063521 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:10.063740 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:10.063741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:10.063741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:10.063889 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:10.063952 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:10.065560 (+ 1608us) server_negotiation.cc:299] Negotiation successful
0924 23:15:10.065652 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 23:15:10.067432 3010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf: Bootstrap starting.
I20260924 23:15:10.068209 3010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf: Neither blocks nor log segments found. Creating new log.
I20260924 23:15:10.068544 3010 log.cc:824] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf: Log is configured to *not* fsync() on all Append() calls
I20260924 23:15:10.069887 3010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf: No bootstrap required, opened a new log
I20260924 23:15:10.073870 3010 raft_consensus.cc:348] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "811b1b632de8486cbd1eb0d0aabc0242" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } } peers { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } } peers { permanent_uuid: "fe936726ab3744f9b0ae8d71852aa4a8" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } }
I20260924 23:15:10.074185 3010 raft_consensus.cc:729] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f42d5ebbc6f74ceab92b8d68acdfc7cf, State: Initialized, Role: FOLLOWER
I20260924 23:15:10.074339 3010 consensus_queue.cc:260] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "811b1b632de8486cbd1eb0d0aabc0242" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } } peers { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } } peers { permanent_uuid: "fe936726ab3744f9b0ae8d71852aa4a8" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } }
I20260924 23:15:10.075059 2984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "811b1b632de8486cbd1eb0d0aabc0242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf"
I20260924 23:15:10.075183 2984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:15:10.076136 2984 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 811b1b632de8486cbd1eb0d0aabc0242 in term 1.
I20260924 23:15:10.076629 2983 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf [term 1 FOLLOWER]: Refusing update from remote peer 811b1b632de8486cbd1eb0d0aabc0242: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:15:10.076910 3010 sys_catalog.cc:564] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:15:10.079491 3133 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:15:10.081833 3148 sys_catalog.cc:455] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "811b1b632de8486cbd1eb0d0aabc0242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "811b1b632de8486cbd1eb0d0aabc0242" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } } peers { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } } peers { permanent_uuid: "fe936726ab3744f9b0ae8d71852aa4a8" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } } }
I20260924 23:15:10.081929 3148 sys_catalog.cc:458] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:15:10.081979 3148 sys_catalog.cc:455] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 811b1b632de8486cbd1eb0d0aabc0242. Latest consensus state: current_term: 1 leader_uuid: "811b1b632de8486cbd1eb0d0aabc0242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "811b1b632de8486cbd1eb0d0aabc0242" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } } peers { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } } peers { permanent_uuid: "fe936726ab3744f9b0ae8d71852aa4a8" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } } }
I20260924 23:15:10.082024 3148 sys_catalog.cc:458] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:15:10.082054 3148 sys_catalog.cc:455] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "811b1b632de8486cbd1eb0d0aabc0242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "811b1b632de8486cbd1eb0d0aabc0242" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } } peers { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } } peers { permanent_uuid: "fe936726ab3744f9b0ae8d71852aa4a8" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } } }
I20260924 23:15:10.082095 3148 sys_catalog.cc:458] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:15:10.083298 3128 catalog_manager.cc:1370] Generated new certificate authority record
W20260924 23:15:10.083365 3147 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:15:10.083427 3147 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:15:10.084053 3128 catalog_manager.cc:1504] Loading token signing keys...
I20260924 23:15:10.084105 3123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "811b1b632de8486cbd1eb0d0aabc0242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "811b1b632de8486cbd1eb0d0aabc0242" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 38271 } } peers { permanent_uuid: "f42d5ebbc6f74ceab92b8d68acdfc7cf" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34615 } } peers { permanent_uuid: "fe936726ab3744f9b0ae8d71852aa4a8" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34365 } } }
I20260924 23:15:10.084231 3123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242 [sys.catalog]: This master's current role is: LEADER
I20260924 23:15:10.086174 3149 mvcc.cc:205] Tried to move back new op lower bound from 7333034844270804992 to 7333034844217774080. Current Snapshot: MvccSnapshot[applied={T|T < 7333034844270804992}]
I20260924 23:15:10.092062 3128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 811b1b632de8486cbd1eb0d0aabc0242: Generated new TSK 0
I20260924 23:15:10.092336 3128 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 23:15:10.119812 2938 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:10.117961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55150 (local address 127.12.212.62:38271)
0924 23:15:10.118009 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:10.118011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:10.119019 (+ 1008us) server_negotiation.cc:407] Connection header received
0924 23:15:10.119131 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:10.119132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:10.119145 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:10.119176 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:10.119712 (+ 536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55150: BlockingRecv error: recv got EOF from 127.0.0.1:55150 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260924 23:15:10.120673 3015 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:10.118393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59258 (local address 127.12.212.61:34615)
0924 23:15:10.118436 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:10.118438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:10.120087 (+ 1649us) server_negotiation.cc:407] Connection header received
0924 23:15:10.120189 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:10.120190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:10.120202 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:10.120232 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:10.120599 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59258: BlockingRecv error: recv got EOF from 127.0.0.1:59258 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 23:15:10.121766 3090 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:10.118725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52182 (local address 127.12.212.60:34365)
0924 23:15:10.118780 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:10.118782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:10.121164 (+ 2382us) server_negotiation.cc:407] Connection header received
0924 23:15:10.121274 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:10.121276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:10.121289 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:10.121318 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:10.121695 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52182: BlockingRecv error: recv got EOF from 127.0.0.1:52182 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260924 23:15:10.124678 3015 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:10.123755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59262 (local address 127.12.212.61:34615)
0924 23:15:10.123800 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:10.123802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:10.124038 (+ 236us) server_negotiation.cc:407] Connection header received
0924 23:15:10.124159 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:10.124160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:10.124183 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:10.124214 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:10.124616 (+ 402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59262: BlockingRecv error: recv got EOF from 127.0.0.1:59262 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 23:15:10.128145 3015 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:10.127254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59264 (local address 127.12.212.61:34615)
0924 23:15:10.127294 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:10.127297 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:10.127546 (+ 249us) server_negotiation.cc:407] Connection header received
0924 23:15:10.127648 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:10.127649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:10.127661 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:10.127691 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:10.128085 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59264: BlockingRecv error: recv got EOF from 127.0.0.1:59264 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260924 23:15:10.132663 3015 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:10.131664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59278 (local address 127.12.212.61:34615)
0924 23:15:10.131713 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:10.131716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:10.132000 (+ 284us) server_negotiation.cc:407] Connection header received
0924 23:15:10.132117 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:10.132118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:10.132132 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:10.132164 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:10.132586 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59278: BlockingRecv error: recv got EOF from 127.0.0.1:59278 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260924 23:15:10.139122 3015 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:10.138118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59290 (local address 127.12.212.61:34615)
0924 23:15:10.138163 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:10.138165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:10.138443 (+ 278us) server_negotiation.cc:407] Connection header received
0924 23:15:10.138547 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:10.138549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:10.138561 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:10.138591 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:10.139060 (+ 469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59290: BlockingRecv error: recv got EOF from 127.0.0.1:59290 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 23:15:10.149544 3015 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:10.148566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59296 (local address 127.12.212.61:34615)
0924 23:15:10.148610 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:10.148613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:10.148937 (+ 324us) server_negotiation.cc:407] Connection header received
0924 23:15:10.149040 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:10.149042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:10.149054 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:10.149082 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:10.149480 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59296: BlockingRecv error: recv got EOF from 127.0.0.1:59296 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 23:15:10.168056 3015 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:10.167096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59308 (local address 127.12.212.61:34615)
0924 23:15:10.167142 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:10.167145 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:10.167419 (+ 274us) server_negotiation.cc:407] Connection header received
0924 23:15:10.167524 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:10.167525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:10.167538 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:10.167566 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:10.167992 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59308: BlockingRecv error: recv got EOF from 127.0.0.1:59308 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260924 23:15:10.202152 3015 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:10.201480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59314 (local address 127.12.212.61:34615)
0924 23:15:10.201526 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:10.201529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:10.201644 (+ 115us) server_negotiation.cc:407] Connection header received
0924 23:15:10.201694 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:10.201695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:10.201706 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:10.201752 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:10.202089 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59314: BlockingRecv error: recv got EOF from 127.0.0.1:59314 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 23:15:10.268150 3015 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:10.267396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59330 (local address 127.12.212.61:34615)
0924 23:15:10.267455 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:10.267456 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:10.267486 (+ 30us) server_negotiation.cc:407] Connection header received
0924 23:15:10.267558 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:10.267559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:10.267570 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:10.267624 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:10.268086 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59330: BlockingRecv error: recv got EOF from 127.0.0.1:59330 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260924 23:15:10.398149 3015 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:10.397473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59332 (local address 127.12.212.61:34615)
0924 23:15:10.397551 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:10.397552 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:10.397564 (+ 12us) server_negotiation.cc:407] Connection header received
0924 23:15:10.397592 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:10.397593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:10.397603 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:10.397668 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:10.398074 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59332: BlockingRecv error: recv got EOF from 127.0.0.1:59332 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260924 23:15:10.656104 3015 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:10.655469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59338 (local address 127.12.212.61:34615)
0924 23:15:10.655550 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:10.655552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:10.655564 (+ 12us) server_negotiation.cc:407] Connection header received
0924 23:15:10.655593 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:10.655594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:10.655606 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:10.655683 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:10.656031 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59338: BlockingRecv error: recv got EOF from 127.0.0.1:59338 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 23:15:11.016222 3120 catalog_manager.cc:1259] Loaded cluster ID: 6e715f3cff0446a7a36600fa4bfa5e6f
I20260924 23:15:11.016312 3120 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8: loading cluster ID for follower catalog manager: success
I20260924 23:15:11.017125 3120 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8: acquiring CA information for follower catalog manager: success
I20260924 23:15:11.017438 3120 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fe936726ab3744f9b0ae8d71852aa4a8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 23:15:11.083982 3147 catalog_manager.cc:1259] Loaded cluster ID: 6e715f3cff0446a7a36600fa4bfa5e6f
I20260924 23:15:11.084071 3147 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf: loading cluster ID for follower catalog manager: success
I20260924 23:15:11.084973 3147 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf: acquiring CA information for follower catalog manager: success
I20260924 23:15:11.085294 3147 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f42d5ebbc6f74ceab92b8d68acdfc7cf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 23:15:11.172257 3219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:11.169428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59340 (local address 127.12.212.61:34615)
0924 23:15:11.169550 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:11.169552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:11.169566 (+ 14us) server_negotiation.cc:407] Connection header received
0924 23:15:11.169635 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:11.169636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:11.169685 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:11.169777 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:11.169941 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:11.170476 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:11.171403 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:11.171618 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:11.171677 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:11.171924 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:11.172043 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:11.172117 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:11.172157 (+ 40us) server_negotiation.cc:299] Negotiation successful
0924 23:15:11.172181 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":70,"threads_started":1}
I20260924 23:15:11.177219 3225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:11.174487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52184 (local address 127.12.212.60:34365)
0924 23:15:11.174628 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:11.174629 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:11.174639 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:15:11.174674 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:11.174675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:11.174835 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:11.174911 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:11.175098 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:11.175613 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:11.176495 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:11.176706 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:11.176773 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:11.176926 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:11.177050 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:11.177104 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:11.177120 (+ 16us) server_negotiation.cc:299] Negotiation successful
0924 23:15:11.177137 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":68,"threads_started":1}
I20260924 23:15:11.181977 3225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:11.179522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52192 (local address 127.12.212.60:34365)
0924 23:15:11.179557 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:11.179558 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:11.179565 (+ 7us) server_negotiation.cc:407] Connection header received
0924 23:15:11.179587 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:11.179588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:11.179597 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:11.179640 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:11.179767 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:11.180261 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:11.181185 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:11.181481 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:11.181519 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:11.181697 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:11.181732 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:11.181774 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:11.181866 (+ 92us) server_negotiation.cc:299] Negotiation successful
0924 23:15:11.181899 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 23:15:11.182272 3232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:11.179213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55166 (local address 127.12.212.62:38271)
0924 23:15:11.179332 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:11.179334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:11.179480 (+ 146us) server_negotiation.cc:407] Connection header received
0924 23:15:11.179526 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:11.179527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:11.179580 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:11.179654 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:11.179809 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:11.180586 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:11.181342 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:11.181552 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:11.181606 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:11.181832 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:11.181953 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:11.182009 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:11.182160 (+ 151us) server_negotiation.cc:299] Negotiation successful
0924 23:15:11.182188 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":63,"threads_started":1}
I20260924 23:15:11.182374 3219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:11.179206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59350 (local address 127.12.212.61:34615)
0924 23:15:11.179301 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:11.179303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:11.179417 (+ 114us) server_negotiation.cc:407] Connection header received
0924 23:15:11.179510 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:11.179511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:11.179523 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:11.179550 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:11.179720 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:11.180565 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:11.181598 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:11.181890 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:11.181945 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:11.182112 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:11.182145 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:11.182172 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:11.182285 (+ 113us) server_negotiation.cc:299] Negotiation successful
0924 23:15:11.182311 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260924 23:15:11.184073 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 2867
I20260924 23:15:11.188964 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 2942
I20260924 23:15:11.193466 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 3017
2026-09-24T23:15:11Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2764 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 23:15:11 dist-test-slave-m0z6 krb5kdc[3236](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 23:15:11 dist-test-slave-m0z6 krb5kdc[3236](info): set up 2 sockets
Sep 24 23:15:11 dist-test-slave-m0z6 krb5kdc[3236](info): commencing operation
krb5kdc: starting...
W20260924 23:15:12.154870 13136 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/test-user.keytab.
Sep 24 23:15:12 dist-test-slave-m0z6 krb5kdc[3236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T23:15:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T23:15:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.212.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
Entry for principal kudu/127.12.212.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab.
I20260924 23:15:12.273972 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:34807
--webserver_interface=127.12.212.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:35181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:34807,127.12.212.61:34499,127.12.212.60:34543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 23:15:12.355888 3249 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 23:15:12.356122 3249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:15:12.356138 3249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:15:12.356159 3249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:15:12.357298 3249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:15:12.357333 3249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:15:12.357347 3249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:15:12.357357 3249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:15:12.357367 3249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:15:12.358868 3249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:35181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:34807,127.12.212.61:34499,127.12.212.60:34543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.62.keytab
--principal=kudu/127.12.212.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.62:34807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.212.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:15:12.359087 3249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:15:12.359273 3249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:15:12.364058 3255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:12.364074 3257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:12.364073 3254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:15:12.364305 3249 server_base.cc:1034] running on GCE node
I20260924 23:15:12.364547 3249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:15:12.364840 3249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:15:12.365999 3249 hybrid_clock.cc:648] HybridClock initialized: now 1790291712365964 us; error 47 us; skew 500 ppm
Sep 24 23:15:12 dist-test-slave-m0z6 krb5kdc[3236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291712, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:15:12.368916 3249 init.cc:377] Logged in from keytab as kudu/127.12.212.62@KRBTEST.COM (short username kudu)
I20260924 23:15:12.369431 3249 webserver.cc:460] Webserver started at http://127.12.212.62:43277/ using document root <none> and password file <none>
I20260924 23:15:12.369658 3249 fs_manager.cc:359] Metadata directory not provided
I20260924 23:15:12.369719 3249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:15:12.369853 3249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:15:12.371057 3249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-0/data/instance:
uuid: "e3f4cdb1b00146b8965cf538450ae825"
format_stamp: "Formatted at 2026-09-24 23:15:12 on dist-test-slave-m0z6"
server_key: "5594d73893f5e200d5355517f98e02c9"
server_key_iv: "852b6634157d7963b0e2fe7f664a171e"
server_key_version: "encryptionkey@0"
I20260924 23:15:12.371443 3249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-0/wal/instance:
uuid: "e3f4cdb1b00146b8965cf538450ae825"
format_stamp: "Formatted at 2026-09-24 23:15:12 on dist-test-slave-m0z6"
server_key: "5594d73893f5e200d5355517f98e02c9"
server_key_iv: "852b6634157d7963b0e2fe7f664a171e"
server_key_version: "encryptionkey@0"
I20260924 23:15:12.372903 3249 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:15:12.373649 3264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:15:12.373842 3249 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 23:15:12.373943 3249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-0/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-0/wal
uuid: "e3f4cdb1b00146b8965cf538450ae825"
format_stamp: "Formatted at 2026-09-24 23:15:12 on dist-test-slave-m0z6"
server_key: "5594d73893f5e200d5355517f98e02c9"
server_key_iv: "852b6634157d7963b0e2fe7f664a171e"
server_key_version: "encryptionkey@0"
I20260924 23:15:12.374008 3249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:15:12.385958 3249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:15:12.388274 3249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:15:12.388427 3249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:15:12.392356 3249 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.62:34807
I20260924 23:15:12.392390 3316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.62:34807 every 8 connection(s)
I20260924 23:15:12.392727 3249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-0/data/info.pb
I20260924 23:15:12.393692 3317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:15:12.395330 3317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:15:12.399171 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 3249
I20260924 23:15:12.399266 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-0/wal/instance
I20260924 23:15:12.399479 13136 external_mini_cluster.cc:1428] Setting key 7fbefd12b9dfc82aff1f7f3dd3a428e3
Sep 24 23:15:12 dist-test-slave-m0z6 krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291712, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:15:12.405982 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.396036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:34807 (local address 127.0.0.1:43288)
0924 23:15:12.396210 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.396255 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 23:15:12.396726 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:15:12.397072 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:15:12.397075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:15:12.397340 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:15:12.397527 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:12.397532 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.398671 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.398672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:12.399579 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:12.399583 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.400331 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.400333 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:12.400394 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.400941 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:15:12.400950 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:15:12.402430 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:15:12.404511 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:12.404517 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:12.404519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:12.404718 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:12.404987 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:12.404988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:12.404988 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:12.405067 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:12.405499 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:15:12.405502 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:15:12.405560 (+ 58us) client_negotiation.cc:769] Sending connection context
0924 23:15:12.405627 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 23:15:12.405763 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260924 23:15:12.405988 3320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.396075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43288 (local address 127.12.212.62:34807)
0924 23:15:12.396262 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.396267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.396343 (+ 76us) server_negotiation.cc:407] Connection header received
0924 23:15:12.396896 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.396899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.396922 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.397025 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.397633 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.398510 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.399692 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.400216 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.400332 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.402531 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:12.402539 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:12.402540 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:12.402548 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:12.404417 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.404800 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.404804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.404804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.404923 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.405124 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.405125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.405125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.405433 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:12.405516 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.405682 (+ 166us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.405853 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":155,"thread_start_us":101,"threads_started":1}
I20260924 23:15:12.406828 3317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.212.61@KRBTEST.COM; defaulting to no policy
I20260924 23:15:12.407195 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.407085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:34499 (local address 127.0.0.1:48572)
0924 23:15:12.407129 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.407173 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 23:15:12.407341 3265 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34499: connect: Connection refused (error 111)
I20260924 23:15:12.407756 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.407678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:34499 (local address 127.0.0.1:48584)
0924 23:15:12.407716 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.407741 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260924 23:15:12.407862 3317 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.61:34499: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34499: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.61.keytab.
Entry for principal kudu/127.12.212.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.61.keytab.
I20260924 23:15:12.425987 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.61:34499
--webserver_interface=127.12.212.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.61.keytab
--principal=kudu/127.12.212.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:35181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:34807,127.12.212.61:34499,127.12.212.60:34543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 23:15:12.457123 3317 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } attempt: 1
I20260924 23:15:12.457700 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.457546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:34499 (local address 127.0.0.1:48592)
0924 23:15:12.457620 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.457666 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260924 23:15:12.458092 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.457962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:34499 (local address 127.0.0.1:48608)
0924 23:15:12.458031 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.458065 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260924 23:15:12.458192 3317 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.61:34499: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34499: connect: Connection refused (error 111)
W20260924 23:15:12.508817 3324 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 23:15:12.509069 3324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:15:12.509095 3324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:15:12.509123 3324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:15:12.510388 3324 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:15:12.510443 3324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:15:12.510461 3324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:15:12.510478 3324 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:15:12.510492 3324 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:15:12.512197 3324 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:35181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:34807,127.12.212.61:34499,127.12.212.60:34543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.61.keytab
--principal=kudu/127.12.212.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.61:34499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.212.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:15:12.512478 3324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:15:12.512697 3324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 23:15:12.517517 3329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:15:12.517704 3324 server_base.cc:1034] running on GCE node
W20260924 23:15:12.517546 3330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:12.517510 3332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:15:12.518098 3324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:15:12.518399 3324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:15:12.519543 3324 hybrid_clock.cc:648] HybridClock initialized: now 1790291712519523 us; error 33 us; skew 500 ppm
Sep 24 23:15:12 dist-test-slave-m0z6 krb5kdc[3236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291712, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:15:12.522184 3324 init.cc:377] Logged in from keytab as kudu/127.12.212.61@KRBTEST.COM (short username kudu)
I20260924 23:15:12.522636 3324 webserver.cc:460] Webserver started at http://127.12.212.61:43573/ using document root <none> and password file <none>
I20260924 23:15:12.522861 3324 fs_manager.cc:359] Metadata directory not provided
I20260924 23:15:12.522914 3324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:15:12.523022 3324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:15:12.524121 3324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-1/data/instance:
uuid: "12a4d14e1bc341f19985f580a07dda9a"
format_stamp: "Formatted at 2026-09-24 23:15:12 on dist-test-slave-m0z6"
server_key: "94e1be0d9015e73d4ac2ae5c090a07ff"
server_key_iv: "8f3882c72c23fd2caa51155bad90df35"
server_key_version: "encryptionkey@0"
I20260924 23:15:12.524446 3324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-1/wal/instance:
uuid: "12a4d14e1bc341f19985f580a07dda9a"
format_stamp: "Formatted at 2026-09-24 23:15:12 on dist-test-slave-m0z6"
server_key: "94e1be0d9015e73d4ac2ae5c090a07ff"
server_key_iv: "8f3882c72c23fd2caa51155bad90df35"
server_key_version: "encryptionkey@0"
I20260924 23:15:12.525686 3324 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 23:15:12.526355 3317 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } attempt: 2
I20260924 23:15:12.526609 3339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:15:12.526805 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.526662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:34499 (local address 127.0.0.1:48616)
0924 23:15:12.526734 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.526769 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260924 23:15:12.526832 3324 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 23:15:12.526927 3324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-1/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-1/wal
uuid: "12a4d14e1bc341f19985f580a07dda9a"
format_stamp: "Formatted at 2026-09-24 23:15:12 on dist-test-slave-m0z6"
server_key: "94e1be0d9015e73d4ac2ae5c090a07ff"
server_key_iv: "8f3882c72c23fd2caa51155bad90df35"
server_key_version: "encryptionkey@0"
I20260924 23:15:12.527005 3324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:15:12.527153 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.527043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:34499 (local address 127.0.0.1:48618)
0924 23:15:12.527085 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.527123 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260924 23:15:12.527236 3317 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.61:34499: Network error: Client connection negotiation failed: client connection to 127.12.212.61:34499: connect: Connection refused (error 111)
I20260924 23:15:12.539309 3324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:15:12.542899 3324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:15:12.543094 3324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:15:12.547048 3324 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.61:34499
I20260924 23:15:12.547122 3391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.61:34499 every 8 connection(s)
I20260924 23:15:12.547461 3324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-1/data/info.pb
I20260924 23:15:12.548520 3392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:15:12.550078 3392 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:15:12.551225 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 3324
I20260924 23:15:12.551329 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-1/wal/instance
I20260924 23:15:12.551481 13136 external_mini_cluster.cc:1428] Setting key becb9427ba3fcd1760e8847623202dd5
WARNING: no policy specified for kudu/127.12.212.60@KRBTEST.COM; defaulting to no policy
Sep 24 23:15:12 dist-test-slave-m0z6 krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291712, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.61@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.212.60@KRBTEST.COM" created.
I20260924 23:15:12.561714 3320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.550816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43296 (local address 127.12.212.62:34807)
0924 23:15:12.550878 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.550880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.552789 (+ 1909us) server_negotiation.cc:407] Connection header received
0924 23:15:12.553335 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.553336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.553350 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.553403 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.554066 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.554604 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.555741 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.555961 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.556020 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.558491 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:12.558493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:12.558493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:12.558502 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:12.559976 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.560503 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.560505 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.560505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.560580 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.560981 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.560983 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.560983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.561131 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:12.561190 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.561566 (+ 376us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.561645 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 23:15:12.561952 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.550819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:34807 (local address 127.0.0.1:43296)
0924 23:15:12.552624 (+ 1805us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.552670 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 23:15:12.553225 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:15:12.553485 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:15:12.553489 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:15:12.553751 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:15:12.553969 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:12.553974 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.554736 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.554737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:12.555651 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:12.555654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.556053 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.556055 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:12.556186 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.556775 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:15:12.556785 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:15:12.558395 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:15:12.560066 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:12.560075 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:12.560077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:12.560359 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:12.560680 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:12.560681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:12.560681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:12.560809 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:12.561206 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:15:12.561216 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:15:12.561411 (+ 195us) client_negotiation.cc:769] Sending connection context
0924 23:15:12.561534 (+ 123us) client_negotiation.cc:240] Negotiation successful
0924 23:15:12.561715 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1755,"thread_start_us":124,"threads_started":1}
I20260924 23:15:12.562674 3392 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 23:15:12 dist-test-slave-m0z6 krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291712, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.61@KRBTEST.COM for kudu/127.12.212.61@KRBTEST.COM
I20260924 23:15:12.570767 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.563159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48634 (local address 127.12.212.61:34499)
0924 23:15:12.563339 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.563345 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.563372 (+ 27us) server_negotiation.cc:407] Connection header received
0924 23:15:12.563394 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.563396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.563412 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.563520 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.563914 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.564624 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.565618 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.565960 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.566024 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.568093 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:12.568105 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:12.568106 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:12.568113 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:12.569632 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.569960 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.569963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.569963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.570013 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.570174 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.570175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.570175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.570339 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:12.570411 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.570528 (+ 117us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.570608 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
I20260924 23:15:12.570771 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.563074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:34499 (local address 127.0.0.1:48634)
0924 23:15:12.563142 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.563153 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:15:12.563199 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:15:12.563557 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:15:12.563559 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:15:12.563716 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:15:12.563783 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:12.563785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.564720 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.564721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:12.565516 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:12.565519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.566039 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.566040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:12.566115 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.566565 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:15:12.566570 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:15:12.568019 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:15:12.569715 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:12.569716 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:12.569717 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:12.569885 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:12.570066 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:12.570066 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:12.570067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:12.570125 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:12.570400 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:15:12.570401 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:15:12.570439 (+ 38us) client_negotiation.cc:769] Sending connection context
0924 23:15:12.570499 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 23:15:12.570671 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":76}
I20260924 23:15:12.571671 3392 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:15:12.572245 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.572104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34543 (local address 127.0.0.1:40062)
0924 23:15:12.572162 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.572217 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260924 23:15:12.572409 3340 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34543: connect: Connection refused (error 111)
I20260924 23:15:12.572824 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.572720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34543 (local address 127.0.0.1:40068)
0924 23:15:12.572774 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.572800 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260924 23:15:12.572923 3392 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.60:34543: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34543: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.212.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.60.keytab.
Entry for principal kudu/127.12.212.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.60.keytab.
I20260924 23:15:12.580196 13136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmL_OPf/build/release/bin/kudu
/tmp/dist-test-taskmL_OPf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.60:34543
--webserver_interface=127.12.212.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.60.keytab
--principal=kudu/127.12.212.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.212.20:35181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.212.62:34807,127.12.212.61:34499,127.12.212.60:34543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 23:15:12.618419 3317 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } attempt: 3
I20260924 23:15:12.622172 3392 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } attempt: 1
I20260924 23:15:12.622605 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.622468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34543 (local address 127.0.0.1:40082)
0924 23:15:12.622526 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.622563 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
Sep 24 23:15:12 dist-test-slave-m0z6 krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291712, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for kudu/127.12.212.61@KRBTEST.COM
I20260924 23:15:12.623524 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.623400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34543 (local address 127.0.0.1:40086)
0924 23:15:12.623451 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.623489 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260924 23:15:12.623644 3392 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.60:34543: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34543: connect: Connection refused (error 111)
I20260924 23:15:12.626106 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.618841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:34499 (local address 127.0.0.1:48648)
0924 23:15:12.618913 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.618931 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 23:15:12.618999 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:15:12.619140 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:15:12.619140 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:15:12.619345 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:15:12.619422 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:12.619425 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.620059 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.620060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:12.620894 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:12.620898 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.621334 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.621335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:12.621411 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.621913 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:15:12.621919 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:15:12.623485 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:15:12.625115 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:12.625116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:12.625117 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:12.625286 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:12.625476 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:12.625477 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:12.625478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:12.625528 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:12.625796 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:15:12.625797 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:15:12.625841 (+ 44us) client_negotiation.cc:769] Sending connection context
0924 23:15:12.625895 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 23:15:12.626002 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260924 23:15:12.626103 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.618933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48648 (local address 127.12.212.61:34499)
0924 23:15:12.618993 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.618995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.619005 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:15:12.619058 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.619058 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.619069 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.619123 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.619492 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.619982 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.621017 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.621258 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.621330 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.623568 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:12.623571 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:12.623571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:12.623581 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:12.625017 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.625366 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.625367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.625367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.625419 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.625594 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.625595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.625595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.625737 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:12.625793 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.625939 (+ 146us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.626014 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 23:15:12.626559 3317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:15:12.626899 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.626813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34543 (local address 127.0.0.1:40090)
0924 23:15:12.626854 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.626879 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260924 23:15:12.627233 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.627079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34543 (local address 127.0.0.1:40104)
0924 23:15:12.627188 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.627211 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
W20260924 23:15:12.627293 3317 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.60:34543: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34543: connect: Connection refused (error 111)
I20260924 23:15:12.658439 3317 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } attempt: 1
I20260924 23:15:12.658965 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.658832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34543 (local address 127.0.0.1:40114)
0924 23:15:12.658895 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.658937 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260924 23:15:12.659317 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.659237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34543 (local address 127.0.0.1:40122)
0924 23:15:12.659275 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.659300 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 23:15:12.659399 3317 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.60:34543: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34543: connect: Connection refused (error 111)
W20260924 23:15:12.665292 3399 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 23:15:12.665517 3399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 23:15:12.665534 3399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 23:15:12.665556 3399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 23:15:12.666805 3399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 23:15:12.666851 3399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 23:15:12.666865 3399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 23:15:12.666877 3399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 23:15:12.666887 3399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 23:15:12.668462 3399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.212.20:35181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.212.62:34807,127.12.212.61:34499,127.12.212.60:34543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/krb5kdc/kudu_127.12.212.60.keytab
--principal=kudu/127.12.212.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.212.60:34543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.212.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 22:10:57 UTC on ccycloud.quasar-rfzvvf.root.comops.site
build id 4671452
I20260924 23:15:12.668728 3399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 23:15:12.668960 3399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 23:15:12.673724 3399 server_base.cc:1034] running on GCE node
W20260924 23:15:12.673681 3406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:12.673692 3408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 23:15:12.673699 3405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 23:15:12.674121 3399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 23:15:12.674396 3399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 23:15:12.675556 3399 hybrid_clock.cc:648] HybridClock initialized: now 1790291712675539 us; error 29 us; skew 500 ppm
Sep 24 23:15:12 dist-test-slave-m0z6 krb5kdc[3236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291712, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 23:15:12.678171 3399 init.cc:377] Logged in from keytab as kudu/127.12.212.60@KRBTEST.COM (short username kudu)
I20260924 23:15:12.678683 3399 webserver.cc:460] Webserver started at http://127.12.212.60:35239/ using document root <none> and password file <none>
I20260924 23:15:12.678937 3399 fs_manager.cc:359] Metadata directory not provided
I20260924 23:15:12.678987 3399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 23:15:12.679107 3399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 23:15:12.680235 3399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-2/data/instance:
uuid: "0b4360138b2b4041a23241bb9ae067f7"
format_stamp: "Formatted at 2026-09-24 23:15:12 on dist-test-slave-m0z6"
server_key: "634e1ec7060933905e1c47e912c8be14"
server_key_iv: "6303b493258f48cd8daee90bb9bf9dda"
server_key_version: "encryptionkey@0"
I20260924 23:15:12.680605 3399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-2/wal/instance:
uuid: "0b4360138b2b4041a23241bb9ae067f7"
format_stamp: "Formatted at 2026-09-24 23:15:12 on dist-test-slave-m0z6"
server_key: "634e1ec7060933905e1c47e912c8be14"
server_key_iv: "6303b493258f48cd8daee90bb9bf9dda"
server_key_version: "encryptionkey@0"
I20260924 23:15:12.682061 3399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 23:15:12.682915 3415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 23:15:12.683096 3399 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 23:15:12.683187 3399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-2/data,/tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-2/wal
uuid: "0b4360138b2b4041a23241bb9ae067f7"
format_stamp: "Formatted at 2026-09-24 23:15:12 on dist-test-slave-m0z6"
server_key: "634e1ec7060933905e1c47e912c8be14"
server_key_iv: "6303b493258f48cd8daee90bb9bf9dda"
server_key_version: "encryptionkey@0"
I20260924 23:15:12.683261 3399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 23:15:12.691838 3392 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } attempt: 2
I20260924 23:15:12.692348 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.692188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34543 (local address 127.0.0.1:40130)
0924 23:15:12.692254 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.692307 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260924 23:15:12.692777 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.692671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34543 (local address 127.0.0.1:40136)
0924 23:15:12.692716 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.692753 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260924 23:15:12.692929 3392 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.212.60:34543: Network error: Client connection negotiation failed: client connection to 127.12.212.60:34543: connect: Connection refused (error 111)
I20260924 23:15:12.708192 3399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 23:15:12.710242 3399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 23:15:12.710381 3399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 23:15:12.714164 3399 rpc_server.cc:307] RPC server started. Bound to: 127.12.212.60:34543
I20260924 23:15:12.714215 3467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.212.60:34543 every 8 connection(s)
I20260924 23:15:12.714546 3399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-2/data/info.pb
I20260924 23:15:12.715534 3468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 23:15:12.715714 13136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmL_OPf/build/release/bin/kudu as pid 3399
I20260924 23:15:12.715874 13136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmL_OPf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790291444018253-13136-0/minicluster-data/master-2/wal/instance
I20260924 23:15:12.716065 13136 external_mini_cluster.cc:1428] Setting key 496434ed2c2319ba74366dc338e2943e
I20260924 23:15:12.717541 3468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 23:15:12 dist-test-slave-m0z6 krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291712, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.60@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
Sep 24 23:15:12 dist-test-slave-m0z6 krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.60@KRBTEST.COM
I20260924 23:15:12.725881 3320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.718246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43304 (local address 127.12.212.62:34807)
0924 23:15:12.718306 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.718308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.718435 (+ 127us) server_negotiation.cc:407] Connection header received
0924 23:15:12.718480 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.718481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.718491 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.718545 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.719050 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.719698 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.720539 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.720740 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.720794 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.722980 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:12.722982 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:12.722982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:12.722990 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:12.724341 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.724782 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.724783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.724784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.724835 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.725102 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.725103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.725104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.725258 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:12.725332 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.725703 (+ 371us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.725800 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 23:15:12.726078 3472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.718206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.62:34807 (local address 127.0.0.1:43304)
0924 23:15:12.718339 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.718377 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 23:15:12.718438 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:15:12.718568 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:15:12.718570 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:15:12.718865 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:15:12.718967 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:12.718972 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.719775 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.719776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:12.720461 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:12.720464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.720827 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.720828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:12.720896 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.721441 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:15:12.721449 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:15:12.722892 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:15:12.724417 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:12.724423 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:12.724425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:12.724718 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:12.724890 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:12.724891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:12.724891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:12.725024 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:12.725350 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:15:12.725356 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:15:12.725539 (+ 183us) client_negotiation.cc:769] Sending connection context
0924 23:15:12.725654 (+ 115us) client_negotiation.cc:240] Negotiation successful
0924 23:15:12.725809 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
I20260924 23:15:12.726619 3468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:15:12.726732 3471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.716565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40150 (local address 127.12.212.60:34543)
0924 23:15:12.716749 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.716762 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.716798 (+ 36us) server_negotiation.cc:407] Connection header received
0924 23:15:12.717302 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.717312 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.717392 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.717539 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.718179 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.719484 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.720352 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.720666 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.720761 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.723342 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:12.723350 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:12.723350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:12.723361 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:12.725029 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.725435 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.725438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.725439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.725504 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.725763 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.725764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.725764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.725924 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:12.726460 (+ 536us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.726508 (+ 48us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.726587 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":82,"threads_started":1}
Sep 24 23:15:12 dist-test-slave-m0z6 krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291712, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.60@KRBTEST.COM for kudu/127.12.212.61@KRBTEST.COM
Sep 24 23:15:12 dist-test-slave-m0z6 krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.62@KRBTEST.COM
I20260924 23:15:12.734584 3472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.726920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.61:34499 (local address 127.0.0.1:48652)
0924 23:15:12.726969 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.726981 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 23:15:12.727024 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:15:12.727131 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:15:12.727132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:15:12.727271 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:15:12.727342 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:12.727344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.728065 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.728066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:12.728722 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:12.728725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.729256 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.729257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:12.729345 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.729878 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:15:12.729884 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:15:12.731515 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:15:12.733408 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:12.733409 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:12.733410 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:12.733612 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:12.733811 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:12.733812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:12.733812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:12.733936 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:12.734226 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:15:12.734227 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:15:12.734288 (+ 61us) client_negotiation.cc:769] Sending connection context
0924 23:15:12.734391 (+ 103us) client_negotiation.cc:240] Negotiation successful
0924 23:15:12.734484 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 23:15:12.734536 3317 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } attempt: 2
I20260924 23:15:12.734601 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.726989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48652 (local address 127.12.212.61:34499)
0924 23:15:12.727030 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.727032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.727054 (+ 22us) server_negotiation.cc:407] Connection header received
0924 23:15:12.727078 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.727078 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.727088 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.727129 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.727412 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.727988 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.728809 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.729174 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.729226 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.731637 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:12.731639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:12.731640 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:12.731647 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:12.733298 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.733681 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.733683 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.733683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.733747 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.733999 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.734000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.734000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.734150 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:12.734220 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.734425 (+ 205us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.734507 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 23:15:12.735404 3468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } has no permanent_uuid. Determining permanent_uuid...
I20260924 23:15:12.735854 3320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.727673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.12.212.62:34807)
0924 23:15:12.727711 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.727713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.727752 (+ 39us) server_negotiation.cc:407] Connection header received
0924 23:15:12.727809 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.727810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.727819 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.727861 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.728460 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.728997 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.729885 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.730194 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.730294 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.733019 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:12.733021 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:12.733021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:12.733030 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:12.734527 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.735054 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.735056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.735056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.735117 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.735288 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.735289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.735289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.735418 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:12.735645 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.735684 (+ 39us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.735753 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Sep 24 23:15:12 dist-test-slave-m0z6 krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291712, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.62@KRBTEST.COM for kudu/127.12.212.60@KRBTEST.COM
Sep 24 23:15:12 dist-test-slave-m0z6 krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291712, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.60@KRBTEST.COM for kudu/127.12.212.60@KRBTEST.COM
Sep 24 23:15:12 dist-test-slave-m0z6 krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.212.61@KRBTEST.COM
I20260924 23:15:12.742319 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.734821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34543 (local address 127.0.0.1:40164)
0924 23:15:12.734872 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.734881 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:15:12.734917 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:15:12.735070 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:15:12.735071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:15:12.735217 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:15:12.735287 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:12.735289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.736182 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.736183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:12.736827 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:12.736831 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.737248 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.737250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:12.737309 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.737709 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:15:12.737714 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:15:12.739417 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:15:12.741065 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:12.741066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:12.741066 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:12.741262 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:12.741485 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:12.741487 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:12.741487 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:12.741559 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:12.741919 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:15:12.741920 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:15:12.741984 (+ 64us) client_negotiation.cc:769] Sending connection context
0924 23:15:12.742085 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 23:15:12.742216 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 23:15:12.742306 3471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.734924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40164 (local address 127.12.212.60:34543)
0924 23:15:12.734961 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.734962 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.734969 (+ 7us) server_negotiation.cc:407] Connection header received
0924 23:15:12.734987 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.734987 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.734996 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.735355 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.735389 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.736079 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.736918 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.737181 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.737238 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.739510 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:12.739511 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:12.739512 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:12.739518 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:12.740969 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.741332 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.741333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.741333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.741401 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.741656 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.741658 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.741658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.741815 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:12.741895 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.742116 (+ 221us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.742204 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 23:15:12.744261 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.736764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48658 (local address 127.12.212.61:34499)
0924 23:15:12.736832 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.736834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.736867 (+ 33us) server_negotiation.cc:407] Connection header received
0924 23:15:12.736937 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.736938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.736948 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.736989 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.737545 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.738275 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.738996 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.739215 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.739271 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.741551 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:12.741552 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:12.741553 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:12.741560 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:12.743179 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.743545 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.743547 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.743547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.743613 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.743765 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.743766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.743766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.743896 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:12.744062 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.744099 (+ 37us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.744174 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 23:15:12.744992 3317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825: Bootstrap starting.
I20260924 23:15:12.745743 13136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 23:15:12.745786 3317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825: Neither blocks nor log segments found. Creating new log.
I20260924 23:15:12.746110 3317 log.cc:824] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825: Log is configured to *not* fsync() on all Append() calls
I20260924 23:15:12.746980 3317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825: No bootstrap required, opened a new log
I20260924 23:15:12.747401 3472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.735677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34543 (local address 127.0.0.1:40178)
0924 23:15:12.735932 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.735941 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 23:15:12.735981 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:15:12.736117 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:15:12.736118 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:15:12.736278 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:15:12.736314 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:12.736317 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.737864 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.737865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:12.738428 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:12.738431 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.738947 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.738948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:12.738995 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.739340 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:15:12.739345 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:15:12.740624 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:15:12.742858 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:12.742858 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:12.742859 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:12.743190 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:12.746581 (+ 3391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:12.746581 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:12.746582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:12.746644 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:12.747110 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:15:12.747111 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:15:12.747150 (+ 39us) client_negotiation.cc:769] Sending connection context
0924 23:15:12.747205 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 23:15:12.747309 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242}
I20260924 23:15:12.746680 3291 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e3f4cdb1b00146b8965cf538450ae825"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43312
I20260924 23:15:12.747707 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.735733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40178 (local address 127.12.212.60:34543)
0924 23:15:12.735893 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.735895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.736015 (+ 120us) server_negotiation.cc:407] Connection header received
0924 23:15:12.736036 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.736036 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.736048 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.736106 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.736397 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.737789 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.738511 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.738842 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.738900 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.740725 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:12.740726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:12.740726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:12.740733 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:12.742779 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.746422 (+ 3643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.746424 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.746424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.746502 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.746738 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.746740 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.746740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.747011 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:12.747086 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.747519 (+ 433us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.747598 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":91,"threads_started":1}
I20260924 23:15:12.748997 3317 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f4cdb1b00146b8965cf538450ae825" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } } peers { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } } peers { permanent_uuid: "0b4360138b2b4041a23241bb9ae067f7" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } }
I20260924 23:15:12.749125 3468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7: Bootstrap starting.
I20260924 23:15:12.749141 3317 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3f4cdb1b00146b8965cf538450ae825, State: Initialized, Role: FOLLOWER
I20260924 23:15:12.749229 3317 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f4cdb1b00146b8965cf538450ae825" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } } peers { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } } peers { permanent_uuid: "0b4360138b2b4041a23241bb9ae067f7" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } }
I20260924 23:15:12.749511 3475 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f4cdb1b00146b8965cf538450ae825" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } } peers { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } } peers { permanent_uuid: "0b4360138b2b4041a23241bb9ae067f7" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } } }
I20260924 23:15:12.749570 3317 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:15:12.749600 3475 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:15:12.749651 3291 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 23:15:12.749711 3291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:15:12.749845 3468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7: Neither blocks nor log segments found. Creating new log.
I20260924 23:15:12.750152 3468 log.cc:824] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7: Log is configured to *not* fsync() on all Append() calls
I20260924 23:15:12.750470 3291 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f4cdb1b00146b8965cf538450ae825" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } } peers { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } } peers { permanent_uuid: "0b4360138b2b4041a23241bb9ae067f7" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } }
I20260924 23:15:12.750742 3291 leader_election.cc:288] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [CANDIDATE]: Term 1 election: Requested vote from peers 12a4d14e1bc341f19985f580a07dda9a (127.12.212.61:34499), 0b4360138b2b4041a23241bb9ae067f7 (127.12.212.60:34543)
I20260924 23:15:12.751055 3468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7: No bootstrap required, opened a new log
I20260924 23:15:12.753145 3468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f4cdb1b00146b8965cf538450ae825" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } } peers { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } } peers { permanent_uuid: "0b4360138b2b4041a23241bb9ae067f7" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } }
I20260924 23:15:12.753306 3468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b4360138b2b4041a23241bb9ae067f7, State: Initialized, Role: FOLLOWER
I20260924 23:15:12.753386 3468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f4cdb1b00146b8965cf538450ae825" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } } peers { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } } peers { permanent_uuid: "0b4360138b2b4041a23241bb9ae067f7" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } }
I20260924 23:15:12.753706 3468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:15:12.753793 3442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3f4cdb1b00146b8965cf538450ae825" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0b4360138b2b4041a23241bb9ae067f7"
I20260924 23:15:12.753913 3442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:15:12.754837 3442 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3f4cdb1b00146b8965cf538450ae825 in term 1.
I20260924 23:15:12.755059 3265 leader_election.cc:302] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b4360138b2b4041a23241bb9ae067f7, e3f4cdb1b00146b8965cf538450ae825; no voters:
I20260924 23:15:12.755187 3475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 23:15:12.752440 3249 master_runner.cc:186] Error getting master registration for 127.12.212.61:34499: OK, instance_id { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" instance_seqno: 1790291712545863 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 23:15:12.755327 3249 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 23:15:12.755394 3495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f4cdb1b00146b8965cf538450ae825" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } } peers { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } } peers { permanent_uuid: "0b4360138b2b4041a23241bb9ae067f7" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } } }
I20260924 23:15:12.755484 3495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:15:12.755336 3475 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [term 1 LEADER]: Becoming Leader. State: Replica: e3f4cdb1b00146b8965cf538450ae825, State: Running, Role: LEADER
I20260924 23:15:12.755831 3475 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f4cdb1b00146b8965cf538450ae825" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } } peers { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } } peers { permanent_uuid: "0b4360138b2b4041a23241bb9ae067f7" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } }
I20260924 23:15:12.756342 3475 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3f4cdb1b00146b8965cf538450ae825. Latest consensus state: current_term: 1 leader_uuid: "e3f4cdb1b00146b8965cf538450ae825" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f4cdb1b00146b8965cf538450ae825" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } } peers { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } } peers { permanent_uuid: "0b4360138b2b4041a23241bb9ae067f7" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } } }
I20260924 23:15:12.756433 3475 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [sys.catalog]: This master's current role is: LEADER
W20260924 23:15:12.757208 3506 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:15:12.757269 3506 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260924 23:15:12.758539 3487 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 23:15:12.758589 3487 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 23:15:12.758397 3399 master_runner.cc:186] Error getting master registration for 127.12.212.61:34499: OK, instance_id { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" instance_seqno: 1790291712545863 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 23:15:12.760552 3399 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 23:15:12.760813 3509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 23:15:12.764668 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.752084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48674 (local address 127.12.212.61:34499)
0924 23:15:12.752125 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.752127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.752714 (+ 587us) server_negotiation.cc:407] Connection header received
0924 23:15:12.752807 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.752808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.752819 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.752845 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.753556 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.754253 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.755442 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.756192 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.756231 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.761063 (+ 4832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:12.761064 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:12.761065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:12.761073 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:12.763114 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.763558 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.763560 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.763560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.763619 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.763932 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.763933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.763933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.764070 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:12.764164 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.764484 (+ 320us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.764574 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 23:15:12.766522 3320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.752302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43326 (local address 127.12.212.62:34807)
0924 23:15:12.752352 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.752354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.752639 (+ 285us) server_negotiation.cc:407] Connection header received
0924 23:15:12.752776 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.752777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.752801 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.752829 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.757073 (+ 4244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.757825 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.758833 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.759125 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.759163 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.763854 (+ 4691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:12.763855 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:12.763855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:12.763863 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:12.765376 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.765723 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.765724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.765724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.765775 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.765962 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.765962 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.765963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.766083 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:12.766170 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.766368 (+ 198us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.766443 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 23:15:12.767378 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.754774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40188 (local address 127.12.212.60:34543)
0924 23:15:12.754809 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.754811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.755672 (+ 861us) server_negotiation.cc:407] Connection header received
0924 23:15:12.755768 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.755769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.755782 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.755844 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.757329 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.758021 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.759541 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.759863 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.759906 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.762061 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:12.762062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:12.762063 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:12.762070 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:12.763596 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.766055 (+ 2459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.766056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.766057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.766121 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.766395 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.766396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.766396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.766535 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:12.766618 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.767159 (+ 541us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.767254 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 23:15:12.768113 3509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 23:15:12.769804 3442 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7 [term 1 FOLLOWER]: Refusing update from remote peer e3f4cdb1b00146b8965cf538450ae825: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:15:12.770152 3507 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b4360138b2b4041a23241bb9ae067f7" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:15:12.771466 3495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3f4cdb1b00146b8965cf538450ae825. Latest consensus state: current_term: 1 leader_uuid: "e3f4cdb1b00146b8965cf538450ae825" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f4cdb1b00146b8965cf538450ae825" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } } peers { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } } peers { permanent_uuid: "0b4360138b2b4041a23241bb9ae067f7" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } } }
I20260924 23:15:12.771564 3495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:15:12.773254 3475 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e3f4cdb1b00146b8965cf538450ae825" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f4cdb1b00146b8965cf538450ae825" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } } peers { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } } peers { permanent_uuid: "0b4360138b2b4041a23241bb9ae067f7" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } } }
I20260924 23:15:12.773340 3475 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [sys.catalog]: This master's current role is: LEADER
I20260924 23:15:12.773442 3513 mvcc.cc:205] Tried to move back new op lower bound from 7333034855502610432 to 7333034855448768512. Current Snapshot: MvccSnapshot[applied={T|T < 7333034855502610432}]
I20260924 23:15:12.773511 3495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e3f4cdb1b00146b8965cf538450ae825" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f4cdb1b00146b8965cf538450ae825" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } } peers { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } } peers { permanent_uuid: "0b4360138b2b4041a23241bb9ae067f7" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } } }
I20260924 23:15:12.773576 3495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:15:12.773742 3512 mvcc.cc:205] Tried to move back new op lower bound from 7333034855502610432 to 7333034855448768512. Current Snapshot: MvccSnapshot[applied={T|T < 7333034855502610432}]
I20260924 23:15:12.774511 3509 catalog_manager.cc:1347] Generated new cluster ID: ac159237c89e48f79b298ccc328c708d
I20260924 23:15:12.774554 3509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 23:15:12.784065 3392 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } attempt: 3
I20260924 23:15:12.784148 3509 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 23:15:12.785419 3509 catalog_manager.cc:1504] Loading token signing keys...
Sep 24 23:15:12 dist-test-slave-m0z6 krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790291712, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.212.61@KRBTEST.COM for kudu/127.12.212.60@KRBTEST.COM
I20260924 23:15:12.791898 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.784374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.212.60:34543 (local address 127.0.0.1:40198)
0924 23:15:12.784458 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0924 23:15:12.784469 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 23:15:12.784507 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 23:15:12.784767 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 23:15:12.784768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 23:15:12.784962 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0924 23:15:12.785010 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:12.785013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.785821 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.785822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:12.786614 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 23:15:12.786616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.787082 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.787084 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 23:15:12.787135 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.787628 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 23:15:12.787634 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 23:15:12.789101 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 23:15:12.790876 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:12.790877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:12.790877 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:12.791096 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:12.791265 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 23:15:12.791266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 23:15:12.791266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 23:15:12.791335 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 23:15:12.791575 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 23:15:12.791576 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 23:15:12.791624 (+ 48us) client_negotiation.cc:769] Sending connection context
0924 23:15:12.791685 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 23:15:12.791789 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260924 23:15:12.792264 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.784569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40198 (local address 127.12.212.60:34543)
0924 23:15:12.784611 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.784613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.784620 (+ 7us) server_negotiation.cc:407] Connection header received
0924 23:15:12.784639 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.784640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.784648 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.784707 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.785080 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.785756 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.786679 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.787028 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.787096 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.789184 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 23:15:12.789186 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 23:15:12.789186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 23:15:12.789193 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 23:15:12.790796 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.791166 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.791167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.791167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.791220 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 23:15:12.791382 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 23:15:12.791383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 23:15:12.791383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 23:15:12.791527 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 23:15:12.791584 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.792099 (+ 515us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.792179 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 23:15:12.793784 3392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a: Bootstrap starting.
I20260924 23:15:12.794492 3392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a: Neither blocks nor log segments found. Creating new log.
I20260924 23:15:12.794787 3392 log.cc:824] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a: Log is configured to *not* fsync() on all Append() calls
I20260924 23:15:12.795819 3392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a: No bootstrap required, opened a new log
I20260924 23:15:12.797655 3392 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f4cdb1b00146b8965cf538450ae825" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } } peers { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } } peers { permanent_uuid: "0b4360138b2b4041a23241bb9ae067f7" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } }
I20260924 23:15:12.797784 3392 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12a4d14e1bc341f19985f580a07dda9a, State: Initialized, Role: FOLLOWER
I20260924 23:15:12.797871 3392 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f4cdb1b00146b8965cf538450ae825" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } } peers { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } } peers { permanent_uuid: "0b4360138b2b4041a23241bb9ae067f7" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } }
I20260924 23:15:12.798184 3392 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 23:15:12.798754 3366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3f4cdb1b00146b8965cf538450ae825" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "12a4d14e1bc341f19985f580a07dda9a"
I20260924 23:15:12.798854 3366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a [term 0 FOLLOWER]: Advancing to term 1
I20260924 23:15:12.799436 3366 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3f4cdb1b00146b8965cf538450ae825 in term 1.
I20260924 23:15:12.800258 3365 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a [term 1 FOLLOWER]: Refusing update from remote peer e3f4cdb1b00146b8965cf538450ae825: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 23:15:12.800563 3507 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 23:15:12.803022 3509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825: Generated new TSK 0
I20260924 23:15:12.803251 3509 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 23:15:12.803509 3524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3f4cdb1b00146b8965cf538450ae825" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f4cdb1b00146b8965cf538450ae825" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } } peers { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } } peers { permanent_uuid: "0b4360138b2b4041a23241bb9ae067f7" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } } }
I20260924 23:15:12.803603 3524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:15:12.803658 3524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3f4cdb1b00146b8965cf538450ae825. Latest consensus state: current_term: 1 leader_uuid: "e3f4cdb1b00146b8965cf538450ae825" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f4cdb1b00146b8965cf538450ae825" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } } peers { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } } peers { permanent_uuid: "0b4360138b2b4041a23241bb9ae067f7" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } } }
I20260924 23:15:12.803697 3524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:15:12.803720 3524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e3f4cdb1b00146b8965cf538450ae825" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f4cdb1b00146b8965cf538450ae825" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } } peers { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } } peers { permanent_uuid: "0b4360138b2b4041a23241bb9ae067f7" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } } }
I20260924 23:15:12.803753 3524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a [sys.catalog]: This master's current role is: FOLLOWER
I20260924 23:15:12.804502 3525 mvcc.cc:205] Tried to move back new op lower bound from 7333034855558463488 to 7333034855448768512. Current Snapshot: MvccSnapshot[applied={T|T < 7333034855502610432}]
I20260924 23:15:12.805500 3507 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e3f4cdb1b00146b8965cf538450ae825" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f4cdb1b00146b8965cf538450ae825" member_type: VOTER last_known_addr { host: "127.12.212.62" port: 34807 } } peers { permanent_uuid: "12a4d14e1bc341f19985f580a07dda9a" member_type: VOTER last_known_addr { host: "127.12.212.61" port: 34499 } } peers { permanent_uuid: "0b4360138b2b4041a23241bb9ae067f7" member_type: VOTER last_known_addr { host: "127.12.212.60" port: 34543 } } }
I20260924 23:15:12.805598 3507 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3f4cdb1b00146b8965cf538450ae825 [sys.catalog]: This master's current role is: LEADER
I20260924 23:15:12.815495 3533 catalog_manager.cc:1259] Loaded cluster ID: ac159237c89e48f79b298ccc328c708d
I20260924 23:15:12.815562 3533 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a: loading cluster ID for follower catalog manager: success
I20260924 23:15:12.816767 3533 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a: acquiring CA information for follower catalog manager: success
I20260924 23:15:12.817139 3533 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 12a4d14e1bc341f19985f580a07dda9a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260924 23:15:12.825229 3320 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:12.824170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43340 (local address 127.12.212.62:34807)
0924 23:15:12.824216 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.824218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.824540 (+ 322us) server_negotiation.cc:407] Connection header received
0924 23:15:12.824647 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.824648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.824659 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.824688 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.825155 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43340: BlockingRecv error: recv got EOF from 127.0.0.1:43340 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 23:15:12.826427 3397 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:12.825605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48686 (local address 127.12.212.61:34499)
0924 23:15:12.825644 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.825646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.825925 (+ 279us) server_negotiation.cc:407] Connection header received
0924 23:15:12.826018 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.826019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.826029 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.826055 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.826363 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48686: BlockingRecv error: recv got EOF from 127.0.0.1:48686 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 23:15:12.827183 3473 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:12.825812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40204 (local address 127.12.212.60:34543)
0924 23:15:12.825849 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.825851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.826680 (+ 829us) server_negotiation.cc:407] Connection header received
0924 23:15:12.826797 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.826798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.826807 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.826835 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.827120 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40204: BlockingRecv error: recv got EOF from 127.0.0.1:40204 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 23:15:12.834522 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.830935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48694 (local address 127.12.212.61:34499)
0924 23:15:12.830970 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.830972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.831199 (+ 227us) server_negotiation.cc:407] Connection header received
0924 23:15:12.831304 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.831305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.831361 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.831407 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:12.831618 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.832396 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.833558 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.833857 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.833897 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.834113 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:12.834253 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:12.834302 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.834421 (+ 119us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.834443 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260924 23:15:12.837806 3473 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:12.837032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40214 (local address 127.12.212.60:34543)
0924 23:15:12.837072 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.837074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.837239 (+ 165us) server_negotiation.cc:407] Connection header received
0924 23:15:12.837356 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.837357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.837378 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.837406 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.837749 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40214: BlockingRecv error: recv got EOF from 127.0.0.1:40214 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 23:15:12.843178 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.840075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48700 (local address 127.12.212.61:34499)
0924 23:15:12.840114 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.840116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.840319 (+ 203us) server_negotiation.cc:407] Connection header received
0924 23:15:12.840422 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.840423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.840433 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.840458 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:12.840612 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.841323 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.842381 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.842668 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.842737 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.842924 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:12.842959 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:12.842982 (+ 23us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.843069 (+ 87us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.843090 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260924 23:15:12.846422 3473 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:12.845673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40218 (local address 127.12.212.60:34543)
0924 23:15:12.845713 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.845715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.845891 (+ 176us) server_negotiation.cc:407] Connection header received
0924 23:15:12.845979 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.845980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.845991 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.846017 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.846368 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40218: BlockingRecv error: recv got EOF from 127.0.0.1:40218 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 23:15:12.853040 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.849692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48708 (local address 127.12.212.61:34499)
0924 23:15:12.849750 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.849752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.850000 (+ 248us) server_negotiation.cc:407] Connection header received
0924 23:15:12.850104 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.850105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.850117 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.850146 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:12.850351 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.851083 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.852226 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.852526 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.852571 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.852777 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:12.852813 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:12.852840 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.852940 (+ 100us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.852961 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 23:15:12.856434 3473 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:12.855630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40228 (local address 127.12.212.60:34543)
0924 23:15:12.855672 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.855674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.855851 (+ 177us) server_negotiation.cc:407] Connection header received
0924 23:15:12.855941 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.855941 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.855952 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.855979 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.856377 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40228: BlockingRecv error: recv got EOF from 127.0.0.1:40228 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 23:15:12.865145 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.861803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48712 (local address 127.12.212.61:34499)
0924 23:15:12.861849 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.861851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.862105 (+ 254us) server_negotiation.cc:407] Connection header received
0924 23:15:12.862213 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.862214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.862226 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.862255 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:12.862464 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.863232 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.864321 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.864607 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.864653 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.864869 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:12.864908 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:12.864935 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.865039 (+ 104us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.865063 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260924 23:15:12.868628 3473 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:12.867715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40244 (local address 127.12.212.60:34543)
0924 23:15:12.867758 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.867760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.867985 (+ 225us) server_negotiation.cc:407] Connection header received
0924 23:15:12.868100 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.868101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.868114 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.868153 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.868563 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40244: BlockingRecv error: recv got EOF from 127.0.0.1:40244 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 23:15:12.881631 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.878058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48714 (local address 127.12.212.61:34499)
0924 23:15:12.878114 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.878117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.878399 (+ 282us) server_negotiation.cc:407] Connection header received
0924 23:15:12.878509 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.878510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.878522 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.878550 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:12.878785 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.879574 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.880780 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.881073 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.881117 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.881347 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:12.881387 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:12.881414 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.881521 (+ 107us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.881544 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 23:15:12.885037 3473 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:12.884182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40258 (local address 127.12.212.60:34543)
0924 23:15:12.884224 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.884226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.884447 (+ 221us) server_negotiation.cc:407] Connection header received
0924 23:15:12.884546 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.884547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.884560 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.884590 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.884972 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40258: BlockingRecv error: recv got EOF from 127.0.0.1:40258 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 23:15:12.905910 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.902495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48726 (local address 127.12.212.61:34499)
0924 23:15:12.902540 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.902542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.902865 (+ 323us) server_negotiation.cc:407] Connection header received
0924 23:15:12.902973 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.902975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.902987 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.903015 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:12.903205 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.903959 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.905104 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.905396 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.905442 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.905640 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:12.905676 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:12.905702 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.905803 (+ 101us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.905825 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 23:15:12.909042 3473 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:12.908214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40274 (local address 127.12.212.60:34543)
0924 23:15:12.908257 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.908259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.908461 (+ 202us) server_negotiation.cc:407] Connection header received
0924 23:15:12.908549 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.908550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.908561 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.908588 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.908982 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40274: BlockingRecv error: recv got EOF from 127.0.0.1:40274 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 23:15:12.944670 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:12.942373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48734 (local address 127.12.212.61:34499)
0924 23:15:12.942421 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.942423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.942569 (+ 146us) server_negotiation.cc:407] Connection header received
0924 23:15:12.942618 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.942619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.942630 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.942677 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:12.942818 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.943293 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.944062 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:12.944246 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:12.944300 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:12.944438 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:12.944468 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:12.944507 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:12.944567 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 23:15:12.944587 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 23:15:12.947131 3473 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:12.946460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40286 (local address 127.12.212.60:34543)
0924 23:15:12.946503 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:12.946505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:12.946623 (+ 118us) server_negotiation.cc:407] Connection header received
0924 23:15:12.946670 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:12.946671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:12.946683 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:12.946755 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:12.947069 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40286: BlockingRecv error: recv got EOF from 127.0.0.1:40286 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 23:15:13.015365 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:13.012378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48736 (local address 127.12.212.61:34499)
0924 23:15:13.012471 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:13.012473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:13.012524 (+ 51us) server_negotiation.cc:407] Connection header received
0924 23:15:13.012575 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:13.012577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:13.012587 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:13.012644 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:13.012980 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:13.013478 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:13.014418 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:13.014628 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:13.014816 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:13.015076 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:13.015114 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:13.015167 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:13.015277 (+ 110us) server_negotiation.cc:299] Negotiation successful
0924 23:15:13.015298 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260924 23:15:13.018430 3473 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:13.017679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40288 (local address 127.12.212.60:34543)
0924 23:15:13.017733 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:13.017735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:13.017795 (+ 60us) server_negotiation.cc:407] Connection header received
0924 23:15:13.017921 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:13.017922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:13.017933 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:13.017982 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:13.018375 (+ 393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40288: BlockingRecv error: recv got EOF from 127.0.0.1:40288 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 23:15:13.150211 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:13.147814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48738 (local address 127.12.212.61:34499)
0924 23:15:13.147907 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:13.147909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:13.147991 (+ 82us) server_negotiation.cc:407] Connection header received
0924 23:15:13.148063 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:13.148064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:13.148075 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:13.148122 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:13.148251 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:13.148768 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:13.149582 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:13.149782 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:13.149838 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:13.149971 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:13.150005 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:13.150046 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:13.150109 (+ 63us) server_negotiation.cc:299] Negotiation successful
0924 23:15:13.150129 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260924 23:15:13.153010 3473 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:13.152300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40292 (local address 127.12.212.60:34543)
0924 23:15:13.152412 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:13.152414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:13.152424 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:15:13.152466 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:13.152467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:13.152480 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:13.152533 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:13.152948 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40292: BlockingRecv error: recv got EOF from 127.0.0.1:40292 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260924 23:15:13.412446 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:13.410158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48750 (local address 127.12.212.61:34499)
0924 23:15:13.410222 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:13.410224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:13.410234 (+ 10us) server_negotiation.cc:407] Connection header received
0924 23:15:13.410319 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:13.410320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:13.410331 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:13.410376 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:13.410502 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:13.411018 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:13.411813 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:13.412008 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:13.412063 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:13.412209 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:13.412247 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:13.412288 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:13.412343 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 23:15:13.412365 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260924 23:15:13.414723 3473 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 23:15:13.414165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40302 (local address 127.12.212.60:34543)
0924 23:15:13.414250 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:13.414251 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:13.414262 (+ 11us) server_negotiation.cc:407] Connection header received
0924 23:15:13.414287 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:13.414288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:13.414298 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:13.414352 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 23:15:13.414636 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40302: BlockingRecv error: recv got EOF from 127.0.0.1:40302 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 23:15:13.757699 3506 catalog_manager.cc:1259] Loaded cluster ID: ac159237c89e48f79b298ccc328c708d
I20260924 23:15:13.757790 3506 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7: loading cluster ID for follower catalog manager: success
I20260924 23:15:13.758594 3506 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7: acquiring CA information for follower catalog manager: success
I20260924 23:15:13.759011 3506 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0b4360138b2b4041a23241bb9ae067f7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 23:15:13.930631 3650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:13.928021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48758 (local address 127.12.212.61:34499)
0924 23:15:13.928162 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:13.928165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:13.928242 (+ 77us) server_negotiation.cc:407] Connection header received
0924 23:15:13.928349 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:13.928349 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:13.928361 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:13.928416 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:13.928550 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:13.929118 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:13.929935 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:13.930156 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:13.930227 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:13.930387 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:13.930421 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:13.930462 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:13.930517 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 23:15:13.930541 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":57,"threads_started":1}
I20260924 23:15:13.934759 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:13.932372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40306 (local address 127.12.212.60:34543)
0924 23:15:13.932471 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:13.932473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:13.932499 (+ 26us) server_negotiation.cc:407] Connection header received
0924 23:15:13.932522 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:13.932523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:13.932569 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:13.932630 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:13.932780 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:13.933322 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:13.934093 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:13.934302 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:13.934345 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:13.934453 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:13.934554 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:13.934606 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:13.934622 (+ 16us) server_negotiation.cc:299] Negotiation successful
0924 23:15:13.934639 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":50,"threads_started":1}
I20260924 23:15:13.939564 3650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:13.936549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48762 (local address 127.12.212.61:34499)
0924 23:15:13.936598 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:13.936600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:13.936609 (+ 9us) server_negotiation.cc:407] Connection header received
0924 23:15:13.936639 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:13.936640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:13.936654 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:13.936711 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:13.936815 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:13.937392 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:13.938513 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:13.938853 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:13.938928 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:13.939169 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:13.939209 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:13.939259 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:13.939450 (+ 191us) server_negotiation.cc:299] Negotiation successful
0924 23:15:13.939480 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 23:15:13.939738 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:13.936485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40312 (local address 127.12.212.60:34543)
0924 23:15:13.936573 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:13.936575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:13.936620 (+ 45us) server_negotiation.cc:407] Connection header received
0924 23:15:13.936673 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:13.936674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:13.936684 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:13.936727 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:13.936845 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:13.937519 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:13.938973 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:13.939204 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:13.939273 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:13.939492 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:13.939527 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:13.939572 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:13.939632 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 23:15:13.939657 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 23:15:13.940083 3663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 23:15:13.936355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43356 (local address 127.12.212.62:34807)
0924 23:15:13.936492 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0924 23:15:13.936495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 23:15:13.936537 (+ 42us) server_negotiation.cc:407] Connection header received
0924 23:15:13.936700 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 23:15:13.936701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 23:15:13.936759 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 23:15:13.936832 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 23:15:13.937054 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:13.937758 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:13.938533 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 23:15:13.938871 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 23:15:13.938934 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 23:15:13.939343 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 23:15:13.939465 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 23:15:13.939525 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 23:15:13.939970 (+ 445us) server_negotiation.cc:299] Negotiation successful
0924 23:15:13.940000 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":54,"threads_started":1}
I20260924 23:15:13.942091 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 3249
I20260924 23:15:13.946949 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 3324
I20260924 23:15:13.951628 13136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmL_OPf/build/release/bin/kudu with pid 3399
2026-09-24T23:15:13Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2758 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5523 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (269940 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST